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FINAL CONTRACT SUMMARY REPORT 


This report summarizes by task the engineering, technical, and management support 
services provided by Vitro Corporation to NASA Headquarters Office of Safety, 
Reliability, Maintainability, and Quality Assurance (now called Office of Safety and 
Mission Assurance (OSMA)) (Code Q) under Contract Number NASW-4311 from the 
period February 22, 1988 through February 10, 1994. 

Each task summary includes significant Vitro accomplishments, conclusions, and 
recommendations for future efforts in each of the following divisions within OSMA: 


Code 

Division Name 

Task Order 

QW 

Engineering and Quality 
Management 

1000, 4000 

QS 

Safety and Risk Management 

2000 

QT 

Payloads and Aeronautics 

3000 

QV 

Software Independent 
Verification and Validation 

5000 

QB 

Quality and Productivity 
Programs Improvement Office 

5000 

QM 

Resources Management 

6000, 7000, 8000 

QP 

Space Flight Safety and Mission 
Assurance 

9000 

DSQ 

SSF Program Office Level II 
Safety and Produce Assurance 
Office 

10000 
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1000 SERIES TASK ORDER 
QUALITY MANAGEMENT 


I. BACKGROUND FOR THE OVERALL TASK 

OSMA is the responsible NASA organization for developing and maintaining leadership 
for safety, reliability, maintainability, and quality assurance disciplines to assure safety 
and mission assurance. This task provides the support necessary in the quality assurance 
functions including technical support for the disciplines of mechanical, electrical, 
chemical, industrial, systems, aerospace, data systems, and quality engineering, and the 
ancillary requirements for effective quality management. This includes functional 
management reviews, audits, preparation and dissemination of policy, procedures, 
handbooks, and guidelines for the effective organization, development, and operation of 
quality assurance programs. These efforts were conducted under Task 1000. The NASA 
Survey/Audit Program Support was initiated under 4300, shifted to 1700, and then to 
1000. 

This task began in 1988, supporting the Reliability, Maintainability and Quality 
Assurance Division. Later this became the Quality Assurance Division, and is currently 
(February 1994) the Engineering and Quality Division. The specific programs under this 
task include the following efforts: 

• Quality Assurance : NASA Handbooks (NHBs) and Management Instructions 
(NMIs) and Department of Defense (DoD) and International Quality System 
Standards. 

• Nondestructive Evaluation (NDE). 

• Metrology and Calibration. 

• Materials and Processes. 

• NASA Mechanical Parts Program Support : Mechanical Parts Control Program, 
Fastener Integrity Program, Fastener Quality Act, Mechanical Information 
Management System (MEPIMS), Government-Industry Information Exchange 
Program (GIDEP), NASA Alert Reporting System (NARS), Supplier Quality 
Research and Technology Operation Plan (RTOP), Contractor Metrics Process 
Action Team (CMP AT), NASA Continual Improvement Implementation Plan 
(CIIP), Assuring Dual Sources for Critical Suppliers, Metric Capabilities of USA 
Machine Shops Survey, Cooperation on Quality Problems with NASA Office of 
Inspector General (OIG), Small Supplier Quality Assurance Assistance (SSQAA) 
Program, Interagency Working Group for Problem Parts and Suppliers, 

International Coordination Meetings. 
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• NASA Survey/ Audit Program Support : Task Initiation, Implementation, Survey 
Activity, FMR Transition. 

The survey subtask evolved through design, implementation, and execution of Field 
Installation on-site surveys by Headquarters, coordination and analysis of corrective 
action reports, and tracking and closure reports for individual surveys. This evolved into 
a process where the Field Installations would conduct local surveys and contractor audits 
and provide data provided to Headquarters for oversight information. With the Field 
Installation agreement to conduct self-surveys, the large Headquarters survey group 
activity would be reduced to conducting only small, focused, spot check reviews. The 
recent development of the NASA "Functional Management" approach to Field 
Installation self-assessments for Safety, Reliability, Maintainability, and Quality 
Assurance (SRM&QA) functional elements will establish accountability at each 
Installation functional manager level. Headquarters will retain oversight responsibility 
and provide Agency level assurance for SRM&QA to the Administrator. The task order 
support function also supports the internal Headquarters Functional Management Task 
Team and related subtask team activity due to Code Q participation and subtask 
coordination functions. The subtask team activity involved revision of the NMI 1240.3 
on Functional Management and development of self-assessment evaluation instruments 
to assess the overall effectiveness of the new Functional Management initiative 
throughout the Agency. 


II. SUMMARY OF THE TASK ACTIVITY 

Vitro assembled an interdisciplinary team with the necessary training, experience, and 
specific expertise to provide innovative, progressive, proactive support to the NASA 
SRM&QA efforts in the task areas. Working closely with their NASA counterparts, 
Vitro team members developed, organized and recommended implementation 
methodologies, new and revised policies, procedures, assessments, technical 
documentation, reviews, and new and restructured working groups to maintain and 
improve the Agency SRM&QA program. Specific details are contained in the task 
summation that follow. 

QUALITY ASSURANCE 

NASA Handbooks (NHBs) and Management Instructions (NMIs) 

The NASA Management Instruction (NMI) 1270.2A, "NASA Quality Policy," was issued 
on October 11, 1991. The revision/rewrite, NMI 1270.2B, was issued on December 22, 
1992. These documents delineated policies and responsibilities for assuring that NASA 
procures, designs, produces/manufactures, maintains, and operates products and services 
that conform to requirements and result in mission success. All Code QR and QW 
programs contain tasks directed towards implementing this NASA Quality Policy. 

Technical procedures, standards, policy, and concepts differ from NASA programs to 
programs and from NASA Field Installation to Installation. Vitro continuously assessed 
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the documentation to identify cross-program generic elements, program-unique 
procedures, and quality elements which could be enhanced/improved. From these 
assessments, recommendations were developed to provide standardized NASA 
Agencywide plans, instructions, and procedures. Vitro has assisted NASA Headquarters 
Code QR/QW in the formulation, generation, coordination, and maintenance of the 
draft NASA publications, as directed. 

The effort to provide a compendium of commonly used safety, reliability, maintainability, 
and quality assurance definitions to ensure standardized assurance communications 
among NASA Field Installations, Headquarters, and contractors culminated with the 
delivery and publication of the NASA Handbook, NHB 5300.4(1G), "NASA Assurance 
Terms and Definitions," in May 1993. This document was the result of extensive 
research into NASA and Department of Defense (DOD) sources/usage of terms, and 
coordination within the Agency. 

As a result of the review of existing NASA Handbooks and the direction to continuously 
improve the NASA procurement procedures and the effective use of resources, an effort 
was initiated to enhance the NASA Handbook, NHB 5300.4(2B-1), "Quality Assurance 
Provisions for Delegated Government Agencies." The requirement to standardize 
management of Government Quality Assurance functions to provide the minimum 
oversight and effective use of resources established the precedence for developing the 
revision of the Handbook. NHB 5300.4(2B-2), "Management of Government Quality 
Assurance Functions for NASA Contracts," which emphasizes active participation by 
NASA and other Agency quality assurance personnel in all phases of contract operations, 
was delivered and issued in April 1993 after coordination with the NASA Field 
Installations and industry. Training was coordinated, conducted, and completed in May 
1993. 

The effort to enhance the NASA Handbook, NHB 5300.4(1B), "Quality Program 
Provisions for Aeronautical and Space System Contractors," was initiated early in the 
contract; however, after extensive coordination with the Headquarters, Installations, 
other Agencies and industry, the final draft rewrite, NHB 5300.4(1B-1), "Quality Program 
Provisions for Aeronautical and Space Systems," was delivered to Code QR and placed 
in a deferred status pending NASA adoption of the American National Standards 
Institute/American Society for Quality Control (ANSI/ASQC) Standards for quality 
systems as augmented by NASA. Vitro maintains the draft document and makes 
distribution when directed. Action was initiated on the associated Handbook, "Guide for 
Corrective Action and Disposition System," but was discontinued at the direction of 
NASA Headquarters. 

In support of the effort to provide program guidance and methodology for structuring 
and administering test programs to ensure successful operation of NASA Aeronautical 
and space systems, Vitro generated and delivered the draft NASA Handbook, NHB 
5300.4(XH), "Test Verification Guidelines for Aeronautical and Space Systems." The 
Handbook is being maintained/updated by Vitro while the NASA-wide coordination 
cycle is underway. 
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Vitro has assisted in the coordination, review, and statusing of the NASA Workmanship 
Standards’ Handbooks, e.g., NHB 5300.4(3G-1), "Requirements for Interconnecting 
Cables, Harnesses, and Wiring, " and NHB 5300.4(3A-3), "Requirements for Soldered 
Electrical Connections." 

Vitro assisted in the preparation/generation and delivery of the NASA Management 
Instructions (NMIs); NMI 1270.2A and NMI 1270.2B, "NASA Quality Policy, NMI 
5320.X, "Protection and Handling of Electrostatic Discharge Sensitive Items" (Draft), and 
NMI 5320.XX, "Configuration Management and Engineering Practices" (Draft). A draft 
"Configuration Management Plan/ Approach" and schedule were developed and delivered 
to Code QR. 

Department of Defense (DOD) and International Quality System Standards 

Vitro assisted in the preparation for NASA participation in the Office of the Assistant 
Secretary of Defense (Production & Logistics) International Organization for 
Standardization, ISO 9000 Working Group. Recommendations were submitted to NASA 
Headquarters for consideration and inclusion in the joint DOD/NASA-HDBK-9000 and 
Data Item Document (DID) to supplement ANSI/ASQC Q91 & Q92 requirements and 
their use in contracts. Vitro generated and delivered the Appendices A1 and A2 for the 
Handbook reflecting NASA unique requirements. 

Vitro also assisted the Headquarters in the preparation of recommendations/ comments 
for the NASA ISO 9000 Working Group which was developing "Augmentation" 
comments for ISO 9001, "Quality Systems - Model for Quality Assurance in Design, 
Development, Production, Installation and Servicing." 

Vitro participated in and assisted in arranging for meetings and discussions, relative to 
the ISO 9000 series of quality standards, with other Government agencies (e.g., the 
General Services Administration (GSA), the US Postal Service, and representatives of 
industry, the Aeronautical Industries Association (AIA), and National Security Industrial 
Association (NSLA). 

General 

Vitro assisted NASA Headquarters, Code QR/QW, in the review and tracking of the 
effort on developing "Critical Skills and Certification" training for NASA Quality 
Engineers, Reliability Engineers, and Quality Assurance Specialists. 

Vitro has participated in and has provided technical and administrative support in 
planning, coordinating, and conducting SR&QA related working group meetings, e.g., 
NASA/AIA/NSIA Liaison Group, the NDE Working Group, Met/Cal Working Group, 
and symposia, e.g., the annual IEEE Reliability and Maintainability Symposia. 

Vitro has supported the review and comment cycle for NASA draft documentation 
submitted by NASA Field Installations, e.g., the "Mission Design Process Guide," the 


4 



SAF/AQXM "New Look" paper, and the draft NMI "Problem/Failure Controls for 
NASA Payloads." Comments and recommendations were submitted to Code QR/QW. 

Vitro’s technical support for the NDE, Metrology and Calibration, and Materials and 
Processes program areas included in the preparation of procedures and policies, 
evaluation of Center and contractor programs, and technical support for technical 
meetings. These activities are described in the following paragraphs. 

NONDESTRUCTIVE EVALUATION 

Vitro also participated in the restructuring of the NASA NDE program through the new 
NDE Working Group. Technical direction was provided in redirecting the large LaRC 
NDE program to be more responsive to NASA program needs. A Code Q committee of 
Center representatives was guided in its formation and operations to provide a NASA- 
wide review and coordination of~RTOP programs and submissions. The NDE program 
area was restructured to provide integrated programs, emphasize customer orientation, 
consensus decisions, and direct application or transition to field problems. 

The multi-million dollar "NDE for Quality Assurance" program at LaRC was brought to 
a completion of the build-up of facilities and personnel resources, and then evaluated in 
terms of changing objectives, approaches, and individual programs in terms of meeting 
stated NASA-wide program needs. This resulted in a broadening of Code Q support to 
other Centers to improve the "Return on Investment" for NDE RTOP programs, which 
had not been satisfactory from the viewpoint of the SRM&QA Directors at the NASA 
Field Installations. 

The Headquarters Oversight Steering Committee was established as a multi-code group 
involving Codes M, Q, R, and S to improve Headquarters communication and 
coordination of NDE efforts. A comprehensive NDE needs survey was conducted to 
provide a basis for new program efforts. The survey was collated and analyzed by a 
Vitro-LaRC team of experts and delivered to the Headquarters NDE Oversight Steering 
Committee. A series of Technical Interchange Meetings was inaugurated on the ViTS 
network for bringing NDE topics and latest state-of-the-art developments to the attention 
of Headquarters and field personnel. The basis for a broad restructuring of the NDE 
program area was established. Technical support and leadership for all of these 
activities was provided under the contract. Stability was provided to the NASA NDE 
community and Headquarters Code Q during a time in which there were four separate 
NDE program managers. The Vitro technical expertise remained constant during the 5- 
year period. 
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A comprehensive review and assessment of the on-orbit NDE and testing requirements 
for the Space Station Freedom program was conducted. This included reviewing the 
requirements stated in the SSP 30000 basic documents, including the WBS and work 
packages, an analysis of requirements and potential NDE techniques suitable for the 
space environment, the types of degradation and failure expected during the planned 30- 
year life of SSF, and a review of instrumentation used in the Russian Salyut series 
vehicles and the Mir space station complex. 

Deliverables: 

"NASA Nondestructive Evaluation (NDE) Working Group Organizational Meeting 
Proceedings," Proceedings of Meeting on April 14-16, 1993 at JSC, September 24, 1993. 

"Nondestructive Evaluation (NDE) Program Restructuring," Status Report, February 22, 
1993. ' — 

"Nondestructive Evaluation Program Restructuring Goals," Planning Report, November 
10, 1992. 

"Acoustic Emission Monitoring NDE Initiative Plan," Status Report, September 9, 1992. 

"Acoustic Emission Monitoring NDE Initiative Plan," Planning and Status Report, July 
22, 1992. 

"NASA Nondestructive Evaluation (NDE) Oversight Steering Committee Charter," 

March 26, 1991. 

"Survey of Nondestructive Evaluation Needs," Technical Report, April 16, 1990. 

"Review of LaRC NDE Plan," Technical Memorandum, December 19, 1989. 

"On-Orbit NDE Assessment [of] Space Station Freedom," Technical Report, March 31, 
1989. 

METROLOGY AND CALIBRATION 

Technical leadership and guidance was provided for broadening the participation and 
interactions of the Metrology and Calibration Working Group, completing the revision of 
the NMI for the program area, quantifying the program goals and milestones for the 
Measurement Assurance Programs, and improving the process of long range planning. 
Technical assistance was provided for revising the "Metrology - Calibration and 
Measurement Processes Guidelines" which is scheduled for completion in 1994. 

Deliverables: 

NASA NMI 5330.9B, "Metrology-Calibration and Measurement Processes," Final Draft, 
July 27, 1993. 
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"Metrology and Calibration Working Group Charter," Final Draft, March 26, 1991. 

MATERIALS AND PROCESSES 

Under this task specific technical surveys and assessments were conducted and reports 
completed. Technical leadership and guidance was provided in the preparation of the 
revision of NASA NHB 8060 to produce NHB 8060. 1C. Numerous drafts of materials 
and processes policy, procedural and process control documents were also prepared and 
distributed to responsible Center materials organizations for review. The policy 
document has been prepared as a final draft and will be submitted for formal review in 
1994. 

Vitro reviewed the materials selection and NDE methods used relating to the orbiter 
fuel door hinge cracking problem. We also assessed test requirements for Kapton 
insulation. NASA requirements and test procedures were compared with those of the 
Aerospace Industry, ASTM, SAE, and the Air Force to determine their ability to provide 
realistic data for selection of Kapton-insulated wire and proposed replacements in the 
Shuttle and other space applications. 

A technical assessment resources study was conducted to compare the personnel 
resources devoted to safety, reliability, maintainability and quality assurance (SRM&QA) 
functions NASA-wide with those of other major Governmental and Industrial 
organizations. This study included over 26,000 NASA employees at Headquarters, 
Centers, and JPL, and compared the results with a survey of over 50 industrial, 
educational, and laboratory organizations with an aggregate of 263,000 employees, and 
non-NASA government agencies, including DOD activities, commissions, laboratories, 
and regulatory agencies with an aggregate of 63,000 employees. This is believed to be 
the largest such survey ever conducted on SRM&QA personnel resources. The NASA 
SRM&QA level of effort did not compare favorably with the overall average for all 
industries (5.3% compared with 6.8%), or with other government agencies (10.6%). The 
NASA resources devoted to SRM&QA showed a marked decline from the 
Apollo/Skylab period (late 1960’s to early 1970’s) to the time of the study (1989). 

Vitro also conducted an analysis of the effects of restrictions proposed by the EPA and 
the Montreal Protocol on major sources and area sources emissions (which would 
include some NASA Center operations) of solvents, CFCs, and other chlorinated 
hydrocarbons. 

Deliverables: 

NMI 8077.XX, "Selection and Control of Materials and Processes for Space Flight 
Programs," July 19, 1993. 

NHB 8060. 1C, "Flammability, Odor, Offgassing, and Compatibility Requirements and 
Test Procedures for Materials in Environments that Support Combustion," Editor and 
Contributor, April 2, 1991. 
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Observations on Environment and Etching Procedures for Orbiter Fuel Door Hinge 
Cracks,” Technical Memorandum, March 19, 1991. 

"Kapton Wire Insulation Test Requirements," Technical Memorandum, December 19, 
1989. 

"SRM&QA Resources Study," Technical Report, February 3, 1989. 

"SRM&QA Manning Study," Interim Technical Report, November 18, 1988. 

"Impact of Proposed Amendments to the Clean Air Act of 1970 on NASA Flight Safety 
Operations," Technical Note, November 2, 1988. 


NASA MECHANICAL PARTS PROGRAM SUPPORT 
Mechanical Parts Control Program 

The mechanical parts control program was established to prevent the problems of 
counterfeit and fraudulent parts and fasteners experienced by NASA and other federal 
agencies during the mid-1980s. Vitro participated in the development, monitoring, and 
research that followed the establishment of the program. The following activities are 
provided as examples: 

Assisted in the establishment of a working group, prepared a charter, and 
monitored the many activities addressed during the time period. 

Worked as part of the team that wrote, coordinated, edited, and delivered for 
publication the NMI 5320.7 "Basic Policy For Mechanical Parts Control." 

Fastener Integrity Program 

The problem of counterfeit and fraudulent fasteners has caused NASA and other 
government agencies to undergo extensive searches, and a large number of fasteners 
were required to be destroyed due to the inability to determine which fasteners were 
genuine and which were misrepresented. This caused NASA to establish an extensive 
fastener integrity program. Vitro was a participant in the following activities supporting 
the Code O organization: 

Assisted in the development of a working group, developed a charter and monitored 
the specific research programs and technical management programs required to 
establish the fastener integrity program. 

Was an active participant in the planning and the establishment of criteria to design 
and develop a Fastener Information Management System (FIMS). 
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Prepared and reviewed a series of position papers on the interpretation of MIL-S- 
8879C and assisted in the preparation of the Code Q policy letter, specifically 
outlining the responsibilities of NASA Centers and Contractors for dimensional 
inspection of both internal and external threaded fasteners. 

Monitored the research on finding a replacement for Cadmium as a coating for 
space and critical ground support equipment. 

Performed literature searches and reviewed company literature for uses of laser 
technology to measure thread and fastener profiles without actual contact. This 
technology has advanced to the point that the measurements can be made, digitized, 
and stored at costs that are competitive with the current methods of establishing 
threaded parts profiles. Several presentations were prepared to make the 
information available to NASA activities. 

Fastener Quality Act 

The Fastener Quality Act (PL 101-592) was enacted into law and a Fastener Advisory 
Panel was established as required by statute. NASA provided a representative to this 
panel and Vitro provided technical and management support for development of 
regulations to implement this law. The NIST was responsible for the publication of the 
regulation. The draft regulations have been circulated, reviewed and commented upon, 
but have not been published. The following facets of the project are identified: 

Reviewed the complete text and background of the law. 

Coordinated NASA comments of the proposed regulations. 

Prepared presentations for the NASA representative. 

Maintained a current status of legislation that proposed specific amendments 
pertinent to the law. 

Mechanical Information Management System (MEPIMS) 

NASA recognized the need to have a system of communication to be a part of the 
mechanical parts control program and established a project at GSFC to develop such a 
system as a part of the NASA Automated System (NAS). Vitro was a vital part of the 
planning, design, development, oversight, and technical review and acceptance. The 
following highlights are listed: 

Assisted in establishing the overall concept and design. 

Provided technical review and oversight of the actual construction and testing. 

Participated in meetings and telecons, and provided comments on technical progress 
reports. 
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Acted as a technical witness for the demonstration and acceptance of the baseline 
system. 

Coordinated with all NASA Centers and prospective users to ascertain what 
functions were required and advised on the prioritization of the development of 
these capabilities. 

Coordinated with the Mechanical Parts Working Group to prepare a plan to accept 
the responsibility for maintaining the system after delivery and to manage the input 
of data and selecting the types of data and capabilities required. 

Government-Industry Information Exchange Program (GIDEP) 

NASA is a sponsor and one of the prime users of the GIDEP system and an active 
participant in the monitoring of the service provided and the improvements to the 
system. Vitro’s participation in all facets of GIDEPs included: 

Acted as custodian for GIDEP Alerts, Safe-Alerts, Agency Action Notices, Problem 
Advisories, Diminishing Manufacturing Sources, and Urgent Information Requests. 

Maintained a computerized summary of all GIDEP Alerts, Safe Alerts, Agency 
Action Notices for rapid response to requests for data. 

Prepared a Monthly Summary of GIDEP data distributed to NASA Centers and 
selected contractors. 

Participated, and frequently hosted, in the GIDEP Management meetings. 

Coordinated with GIDEP representatives on technical aspects of the changes being 
made to the computer system that will allow real time, full search capability for all 
GIDEP data and provided interface with NASA Centers on service problems. 

Acted as Vitro GIDEP representative and performed specialized searches for the 
NASA staff as required. 

Provided the interface between GIDEP personnel and NASA personnel that are 
developing a NASA-wide access to GIDEP and the downloading of GIDEP data. 

Maintained a paper copy file of all incoming GIDEP alerts. 

Assured that a paper copy of all Safe-Alerts was promptly forwarded to Code QS 
for review and possible action. 

NASA Alert Reporting System (NARS) 

NASA recognized the need for an Alert system that would be responsive, closed-loop, 
and closely linked with the GIDEP system. This system is in the final stages of 
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development and will become a part of the NAS electronic data system. Vitro has 
provided a vital input to the development of this system; a few significant examples are 
provided: 

Actively participated in the development of specifications, recommended interface 
techniques, monitored the software design and development, monitored the 
progress, and coordinated with NASA Centers to assure that their needs were 
recognized and met. 

Participated in telecons, working group meetings, progress reviews, and center 
coordination meetings and acted as interface with GIDEP. 

Prepared for the Acceptance Test and will act as technical witness for the 
demonstration of the operational system. 

As a part of the NARS system, NASA has developed a closed-loop NASA Preliminary 
Alert System (NPAS) that gives Code Q a method to rapidly provide NASA Centers with 
sensitive and time-critical information. This system allows electronic or hard-copy 
transmission to selected center representatives for their action as required. The system 
also allows answers to be returned and actions closed when warranted. Vitro was a part 
of the development of this system; selected accomplishments are listed: 

Assisted in the design of the NPAS and coordinated with the center GIDEP 
representatives to assure that needs were being met. 

Reviewed incoming information to determine if it was a candidate for the NPAS. 

Maintained an overview to assure that the information was being received, actions 
taken, and closing reports made. 

Vitro assisted in preparing, coordinating, and distributing the following NMIs and NHB: 

• NMI 53 10. ID, "NASA Alert Reporting of Parts, Materials, and Safety Problems." 

• NMI 5310.2C, "Participation in the Government-Industry Data Exchange Program 
(GIDEP). 

• NHB 5310.3, "Procedures for NASA Alert Reporting of Parts, Materials, and Safety 
Problems" 

Supplier Quality Research And Technology Operation Plan (RTOP) 

A need was recognized by NASA that some system was required to objectively measure 
the performance of suppliers of parts, materials, and services. This RTOP was 
established to research the contractor and NASA community to ascertain how 
evaluations were being performed and how they could be improved to be more objective 
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and standardized between projects and Centers. Vitro has participated in this project, 
and some of the inputs are provided: 

Assisted in developing the guidelines in establishing the project. 

Coordinated with the Lead Center (JSC) in establishing a working group, a 
Statement of Work (SOW), a preliminary plan of action. 

Interacted with the working group to reschedule the project to accept related, more 
critical projects. 

Contractor Metrics Process Action Team (CMP AT) 

Mr. Goldin, NASA Administrator, established a requirement to develop a series of 
metrics for the evaluation of the 30 largest NASA contracts. These metrics were to be 
graphic presentations for use by the Administrator to brief the CEOs of the respective 
companies to show them how they are performing on NASA contracts. Code Q was 
made responsible for the Quality Metric. Vitro provided the following support in this 
effort: 

Attended the CMP AT meetings and at times represented Code Q in the meetings. 

Coordinated with the Lead Center (JSC) to reschedule work on the Supplier 
Quality RTOP and provide inputs for the project. 

Assisted in the development of the Quality Metric, participated in the RTOP 
Working Group meeting teleconferences, reviewed the draft output, and 
coordinated with centers and selected companies on facets of the project. 

Reviewed the proposed presentation technique and provided briefings and 
coordinated with activities that had problems understanding the Oregon Matrix 
technique. 

Coordinated with the CMP AT manager (Code H) to assure that the proper 
documentation was prepared and submitted to the NASA HQ Associate 
Administrators as part of the NHB 2340.4, " Contractor Metrics Handbook." 

NASA Continual Improvement Implementation Plan (CUP) 

NASA established a requirement for a plan to assure that the continual improvement 
concept was exercised by all NASA activities. Code Q was tasked to provide a section to 
the implementation plan on Supplier Quality (Section 6). Vitro assisted in the 
preparation and provided the following support: 

Coordinated with Supplier Quality RTOP leader to reschedule the RTOP activity 
and provide an input for this project. 
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Worked with the working group to prepare the Section 6 input for the CIIP. 

Participated in meetings to assure proper merging of Section 6 into the overall plan. 

Reviewed the Revised Low Award Selection Criteria and made recommendations 
to Code Q. 

Assuring Dual Sources for Critical Suppliers 

NASA was directed by the Office of Management and Budget (OMB) to make an 
annual report on the status of single or endangered sources of supply for critical parts, 
processes, or services. Code Q has been assigned to collect the reports from all NASA 
Program and Project Managers and prepare a consolidated report to OMB. Vitro 
participated in this effort in the following areas: 

Participated in meetings and briefings from Defense Contract Management 
Command (DCMC), a part of Defense Logistics Agency (DLA). 

Assisted in the preparation of briefings to the HQ Program Managers to describe 
the requirements of the report and the possibilities of utilizing support from 
DCMC. 

Reviewed the DCMC program and the data supplied on their current operation in 
performing this task for Department of Defense (DOD). 

Metric Capabilities of USA Machine Shops Survey 

Congress has mandated that all government agencies develop a schedule to convert to 
the use of Metric Measurements as the Primary Unit of Measurement. NASA is 
required to make a report to Congress annually on the progress made in this effort. 
NASA requested that Vitro conduct a survey to determine the current capability of 
machines shops in the USA to provide quality parts in metric dimensions. This survey 
was performed and a report was submitted to Code Q. The results of the study were 
presented at several meetings and briefings. An abstract of the report was published in a 
Metric Measurement Newsletter. 

Cooperation on Quality Problems with NASA Office of Inspector General (OIG) 

OIG notifies Code Q of any critical quality problems that they detect during their 
investigations. These problems are usually sensitive, since the investigation is still being 
performed. Code Q is responsible to review these alleged problems and determine if 
there is a potential for a problem within any NASA program or project. Code Q uses 
the NARS to communicate this information to the proper individuals, yet must protect 
the sensitivity of the information. Vitro assists Code Q in the evaluation of the 
information, preparation of communication to the field, coordination with field activities 
during their research, and the preparation of a report back to OIG advising them of the 
action taken and the result of the investigation if applicable. In providing this assistance, 
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Vitro coordinated with OIG, field representatives, and other agencies to perform 
independent research on some of the suspected individuals and companies. Alleged 
problems included defective ball bearings, ignition testers, pyrotechnic valves, and 
mechanical parts/fasteners. In many of these cases, no problem was detected. 

Small Supplier Quality Assurance Assistance (SSQAA) Program 

NASA has been encouraged by Congress to provide more of their contracts to small and 
disadvantaged companies. The problem that NASA found was that small organizations 
did not understand the quality documentation requirements demanded by NASA, and in 
many cases were reluctant to attempt to establish the required quality controls and 
documentation. In order to assist these prospective government suppliers, NASA 
requested Vitro to develop a process that would be simple, educational, economical, and 
timely to assist the new supplier in understanding and complying with the government 
quality requirements. Vitro took the following actions: 

Developed a plan, prepared a charter for establishment of a working group, 
organized a working group, and scheduled and participated in several meetings with 
NASA representatives, Prime Contractor personnel, Professional Association 
representatives, and Small Business Agency (SBA) personnel. 

« 

Designed, developed, prepared software, created questionnaire, prepared users 
manual, and demonstrated the technique to prospective supplier users. This 
technique was designed to be used on a personal computer, to be very user-friendly, 
and to document the results of the questionnaire directly on a diskette for return 
and inclusion in a data base for future use by the government or large industrial 
customer. The questionnaire was designed to provide the supplier with insight of 
what quality requirements were necessary, and on most occasions why they were 
needed, and how to achieve them. 

After extensive testing within NASA and with the working group member 
organizations, Vitro defined and prepared a final questionnaire, diskette program, 
users manual, and mail-out package. 

Demonstrated and briefed the program at several NASA Small and Disadvantaged 
Business Conferences and provided the attendees with a diskette for their use and 
input to NASA. 

In cooperation with the NASA working group, SBA was convinced of the merit of 
the program, and it was implemented into their Procurement Automated Source 
System (PASS). It is currently available nationwide to any PASS member for direct 
electronic use as a means of evaluating the quality organization and documented 
capability of a prospective small supplier. 
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Interagency Working Group for Problem Parts and Suppliers 

NASA recognized that all Federal agencies were having problems with parts and 
suppliers. They decided to take the lead in the establishment of working group of 
federal agencies to periodically meet and discuss their problems, exchange sensitive 
investigative information, and create a rapport for informal exchange of time-sensitive 
information. Vitro assisted in developing a charter, scheduling meetings, providing 
technical support for documenting the sanitized meeting minutes, and advising working 
group members of NASA points of contacts. The idea was described to OMB and a 
letter was prepared by OMB stating that all federal agencies should participate and that 
GIDEP be tasked to develop a secure special alert and communication system between 
the agency members. 

International Coordination Meetings 

NASA hosted ESA/NASA and NASDA/NASA meetings at Vitro which were attended 
by Vitro representatives. Vitro was tasked to provide meeting minutes and identify all 
action items and follow up on the actions taken. The Mechanical Parts action items 
were tracked and actions documented. 


SURVEY/EVALUATION SUPPORT 

The early efforts required analysis of NASA documented requirements that substantiate 
the Code Q oversight responsibility stated in NMI 1103.39 based on the organizational 
Safety, Reliability, Maintainability, and Quality Assurance (SRM&QA) functional 
requirements. Vitro compiled and delivered to the NASA task manager detailed reviews 
and analyses of NASA Management Directives, Program Directives, Code of Federal 
Regulations, and Federal Acquisition Regulations. One early task required review of all 
previous Engineering Division Reliability and Quality Assurance surveys and Safety 
Division surveys for corrective action closure status and provide results with 
recommendations for Code Q actions. The previous post Challenger accident Code Q 
survey of the Space Shuttle Return to Flight operations at Kennedy Space Center was a 
large review effort with status briefings and requests for corrective action status 
generated to complete the closures. Items remaining of concern to Headquarters were 
identified at each Center and considered for followup review in the new survey cycle. 
During the development of survey plans and schedules, the decision to implement the 
Field Installation on-site surveys was made. The transition was smooth due to NASA- 
contractor coordination; projected planning and implementation schedules were 
developed and approved by NASA. 

The survey planning was implemented by Vitro with a programmed lead time for NASA 
management decisions as to scope, content, manning, timing, and technical support from 
Field Installations not being surveyed. The plan was prepared and the Survey Program 
Schedule was issued by the Associate Administrator to Institutional Program 
Management Codes and Center Directors and Safety, Reliability and Quality Assurance 
Directors. This was an ambitious schedule to survey all Field Installations and the Jet 
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Propulsion Laboratory (JPL) in a 2-year period. The undertaking was well planned and 
executed such that the first survey started as scheduled. The planning, preparation, and 
conduct milestones became the yardstick by which all NASA surveys were organized. 

The standard scenario consisted of scheduling a meeting of the Code Q Executive 
Management Council (Associate Administrator and all Division Directors) to review and 
identify the functional areas to be covered by the survey group and to select the survey 
Team Leaders. The team membership was selected from Headquarters, Field 
Installation, or corporate resources to provide knowledgeable, experienced personnel for 
each of the designated survey areas. 

Vitro provided all coordination support for facsimile notices, telephone conferences, 
video conferences, and various meetings of the Code Q Executive Council, Survey Group 
and Team Leaders. In direct support of Code Q, Vitro accomplished the design and 
development of planning guides and forms, the receipt and distribution of resource 
material requested from the Field Installations, training presentation of administrative 
scenario and data requirements for the survey activity, preparation of draft and final 
individual plans, arrangements for on-site hotel meeting facilities, production of draft and 
final survey records and reports, tracking, analysis, and statusing of corrective action 
reports, and preparation of final closure status. In parallel with the survey planning, a 
task was assigned to create a Headquarters documentation tree relating the Quality 
Assurance management directives, both NMI and NHB, and the Safety management 
directives and federal regulations under Code Q responsibility. The relationship of each 
Center’s management directives to the Headquarters directives was investigated and 
identified. A compilation of the Headquarters and each Center’s management directives 
was obtained, reviewed, maintained, and used to provide copies to individual survey team 
members leading up to preparation for the Center visits. 

Performance of this task during the contract term involved a biennial survey cycle 
scheduling each NASA Installation and JPL with a full baseline review of all active 
SRM&QA functional requirements. The master schedule promulgated by Code Q was 
maintained throughout the 2-year cycle with only one revision. This was a change in 
sequence for the Kennedy Space Center from the middle to the end of the schedule. 

The surveys were coordinated with the NASA Office of Inspector General and the 
resident representative at each Installation. The NASA Institutional Program Office 
Codes were advised by Code Q Associate Administrator letter as to the master schedule 
as well as advance notification before each survey was planned and executed. 

Each survey cycle started with a meeting of the Executive Planning Council consisting of 
the Code Q Associate Administrator and all Division Directors and the designated 
Survey Coordinator. This meeting set the scope of the survey as to Team coverage and 
Areas to be Surveyed and recommended individual Team Leaders. Then, a Team 
Leaders meeting was held with the Chairman of the survey to further define the Areas to 
be Surveyed and select appropriate technical area staff to perform the survey. 

Subsequent survey group meetings were held to define the need for reference material to 
be requested from the Center and to develop tailored checklists which defined the extent 
of the known documented requirements by Federal Law, NASA Directive, and certain 
Center and State regulations. A sample of the various meetings leading up to the on-site 
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visit were videotaped, and a training video was produced for Code Q. This video was 
used to conduct training orientation for new NASA personnel who had not been exposed 
to the survey process during the preparation phase for each survey. 

The survey approach, on-site requirements, and lodging accommodations were 
coordinated by the contractor for the entire survey group. An on-site organizational 
meeting was scheduled for the participants the evening before the start date at the hotel 
to check last minute arrangements and ensure all non-Headquarters participants were 
coordinated with their team leader before arrival at the Center. 

During the on-site survey, the contractor provided a survey coordinator and 
administrative assistant to support the various team members in preparing typed Survey 
Records and access to reference documents. One innovative technique that was 
implemented and used to support a large volume of typing was the use of NASAMail to 
transmit files between Washington, D.C. and the Center. Handwritten drafts were sent 
by facsimile to the contractor facility, typed as individual record files, concatenated into 
one file, and sent via NASAMail addressed to the administrative assistant who 
downloaded the file and printed the final Survey Records for signature by the Center 
Contact, Surveyor, Team Leader, and Chairperson before the survey group departed. 

This provided a clear understanding of the discussion and recommendation entries by 
both Center and Headquarters participants. Within a week after the survey, the draft 
summary report sections were written and formatted by the contractor for review by 
Headquarters and sent to the Center for comment as to technical accuracy. The final 
report was then prepared for issue by the Associate Administrator to the Center, the 
Inspector General Office, and the Institutional Program Office. The Survey Records 
identifying Observations and Findings were organized into data packages and a computer 
data base was developed that provided status reports by various breakdowns. One report 
provided a listing of only Survey Records which were overdue based on the Estimated 
Due Date and was provided to NASA surveyors, team leaders, and the chairperson for 
emphasis to the respective Center. 

A Center-approved SR&QA Corrective Action Plan was requested within 30 days after 
the final report was received and became the basis for the closed loop corrective action 
system. The individual response to each Survey Record was provided by Code Q to the 
contractor, and the assigned Survey Coordinator separated each response by surveyor 
and team and formulated a package for the NASA Survey Chairman to send by memo to 
the survey members. The memo requested the author’s and Team Leader’s assessment 
of the Center response with recommendation as to closing or not closing the Survey 
Record. Survey Corrective Action Plan Status Reports were prepared for response to 
the Center via NASA correspondence. The data base maintained all actions dated from 
conduct of the survey through the date of closure. The data files were updated routinely 
and provided to the NASA Division Director biweekly. The status reports identified 
Center, Date, Team Leader, Survey Record Number, Subject, Corrective Action 
Acceptance or Rejection Date, Estimated Completion Date, C/A Received from Center, 
and Team Leader and Division Director Approval Dates for closure. 
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The Center surveys and respective milestone dates are: 


SRM&QA SURVEY PROGRAM 


NASA 

SURVEY 

FINAL 

RECEIVE 

ACCEPT 

SURVEY 

CENTER 

DATE 

REPORT 

CAP 

CAP 

CLOSED 

JSC/WSTF 

11/89 

1/90 

3/90 

4/90 

8/91 

MSFC 

1/90 

3/90 

6/90 

9/90 

3/92 

SSC 

3/90 

5/90 

6/90 

7/90 

1/92 

MSFC RO 

3/90 

5/90 

7/90 

9/90 

1/91 

JPL 

5/90 

8/90 

10/90 

1/91 

10/92 

GSFC 

8/90 

10/90 

2/91 

5/91 

10/92 

LaRC 

'2/91 

4/91 

5/91 

7/91 

“ 12/93 

LeRC 

5/91 

7/91 

8/91 

10/91 

4/93 

DFRF 

7/91 

9/91 

— 

2/92 

12/93 

ARC 

8/91 

10/91 


2/92 

5/93 

KSC (FMR) 

11/91 

3/92 

5/92 

8/92 

OPEN-1 


In addition to the Survey Report, the Vitro survey coordinator was instrumental in 
consolidating specific commendations and additional information from the surveyors that was 
considered to be a Lesson Learned. These items were drafted and formatted into a Lessons 
Learned Report and distributed to each Center SR&QA office for information and possible 
incorporation as desired. 

All individual Survey Records and Center responses were filed for a permanent NASA 
record. At the termination of this contract, all Center surveys are closed except the last 
survey. The last survey was a Functional Management Review conducted at Kennedy Space 
Center in November 1991, and only one action recommendation is being held open. The 
open item is a large management reorganization and consolidation of the payload assembly 
process documented by both McDonnell Douglas procedures and NASA procedures. These 
procedures are used on the same assembly line by different employees and have different 
requirements and procedural activity that is being consolidated to increase efficiency and 
avoid possible confusion. The estimated completion date is June 1994. 

With the Kennedy Space Center activity beginning the new concept of a Functional 
Management Review (FMR), NASA Code Q began to formulate plans to evolve to functional 
reviews in place of the full SRM&QA Surveys. The NASA Office of the Administrator was 
advancing the reorganization of the Function Management Program previously defined but 
never adequately implemented nor effectively utilized by the large cross-section of NASA 
management. There were at least two Headquarters Task Teams that started the 
reconstruction of the NMI. In the second group, reorganized after a change in senior 
management, the contractor supported both the coordination of the Headquarters Task Team 


18 































































participation by Code Q and the individual Code Q establishment of a Functional 
Management Initiative in collaboration with all the NASA Center SR&QA offices. 

The early effort involved formulating a draft FMR Guidelines strawman document and 
sending it to each Center SR&QA office for comment. There were a couple of telephone 
conferences to discuss the Code Q/Center approach, and the Code Q manager assigned to 
this item was changed three times. The Headquarters Task Team formed a subtask team to 
define the Roles and Responsibilities of each management entity. The contractor support 
continued and the effort expanded to provide coordination for the next Headquarters subtask 
team that was assigned to draft the revised NMI 1240.3 implementing the rewritten roles and 
responsibilities. The review comments and modifications were maintained and provided by 
Code Q to Code J for final approval and distribution. 

During the last two task order periods, the coordination between Headquarters and the 
Centers was solidified to implement a direct approach to the Functional Management Self- 
Assessment Initiative. The Headquarters revision to the Functional Management Program 
was recommended to Vice President Gore’s National Performance Review as an example of 
"Reinventing Government" and was accepted as one of the NASA Laboratories. Code Q is 
using the Functional Management Initiative Working Group to implement the self-assessment 
concept and individual Center accomplishment. The Center implementation plans are used to 
report the Code Q status via Code J to the Administrator. The working group collaboration 
is ongoing and will be pursued rigorously in the future to ensure that the revised program 
achieves the desired success and that programmatic weaknesses are identified and corrected. 

ELECTRICAL, ELECTRONIC, AND ELECTROMECHANICAL (EEE) PARTS 

Vitro personnel have contributed to the development and implementation of an effective 
NASA program for Electrical, Electronic, and Electromechanical (EEE) Parts. (EEE parts 
was a subtask under the Task Order 1000 Series until 1993; see Task Order 4000 for the 
most recent EEE parts activity.) Support has been provided in such areas as: the preparation 
and integration of EEE parts program documentation; organization, support, and participation 
activities for meetings, committees, and working groups; support of the review and 
integration of Research Technical Operating Programs (RTOPs); development and 
preparation of presentations and a film on NASA EEE parts programs and activities; support 
to and participation in field installation surveys; and parts investigations, analyses, reports, 
and database activities. The highlights of the activities performed for the EEE Parts Program 
task are summarized in the following paragraphs. 

During the first year, NHB 5300. 4(1F), "Electrical, Electronic, and Electromechanical (EEE) 
Parts Management and Control Requirements for NASA Space Flight Programs," was 
finalized and issued. Vitro contributed to the preparation of the provisions in that document 
and coordinated its review and approval by NASA field installations and Headquarters codes. 
Subsequently, proposed improvements of key provisions have been prepared and coordinated 
based on Inspector General investigations, changes in the state-of-the-art of EEE parts 
management practices, and discussions with NASA activities, other government agencies, 
and industry associations. 
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Vitro also revised the two key NMIs on EEE Parts. The review, approval, and issuance of 
NMI 5320.5B on the "Basic Policy for Electrical, Electronic, and Electromechanical (EEE) 
Parts” and NMI 5320. 6B on "Implementation of NASA Standard Parts Program" were 
accomplished. These documents provide the basic requirements for the NASA EEE Parts 
Program. 

A working group was organized and monitored to develop new and revised NASA NHBs on 
standard electronic packaging workmanship requirements. Vitro then coordinated the final 
review, approval, and issuance of the documents. These NHBs included "Requirements for 
Soldered Electrical Connections," "Requirements for Interconnecting Cables, Harnesses, and 
Wiring," "Requirements for Crimping and Wire Wrap," "Requirements for Printed Wiring 
Boards," "Requirements for Conformal Coating and Staking of Printed Wiring Board and 
Electronic Assemblies," "Design Requirements for Rigid Printed Wiring Boards and 
Assemblies," "Requirements for Surface Mount Device Assemblies," and "Requirements for 
Control of Electrostatic Sensitive Devices. " 

Efforts were also undertaken to promote the joint development and utilization of documents 
with other government agencies and industry associations. Vitro made numerous 
comparisons of NASA documents with key industry and government documents covering 
similar requirements and contributed to discussions with the AIA, NSIA, and Department of 
Defense to promote joint use of standards for EEE parts management, soldering, and 
electrostatic sensitive devices. Also, efforts were initiated to help develop a revised version 
of the Air Force Space Division MIL-STD-1546 on Requirements for Management of Parts, 
Materials, and Processes which may provide integration of requirements now contained in 
NHB 5300.4(1F). Recently, Vitro contributed to the preparation of a Memorandum of 
Understanding with the Air Force Space Division to jointly utilize standard documents in the 
future. 

Vitro provided key technical and organizational support to meetings of NASA personnel, 
meetings with other government agencies and industry organizations, and meetings with the 
European Space Agency (ESA) and the National Development Space Agency of Japan 
(NASDA) on EEE parts. This included preparing meeting agendas and reports, organizing 
and coordinating the meetings, providing facilities, acting as a member of the NASA 
Headquarters team, and contributing technical advice and consultation. 

Vitro was instrumental in organizing a NASA Parts Steering Committee to provide 
coordination and interchange of information among NASA field installations. Meetings of 
the steering committee were organized and coordinated at Vitro facilities and at Goddard 
Space Flight Center (GSFC), Johnson Space Center (JSC), Marshall Space Flight Center 
(MSFC), Lewis Research Center (LeRC), Langley Research Center (LaRC), and the Jet 
Propulsion Laboratory (JPL). Recently, the importance of the steering committee was 
enhanced by providing it with the responsibility of approving and managing the efforts of the 
Research Technical Operating Programs for EEE parts. 

Key working groups of the NASA Parts Steering Committee were also organized and 
coordinated. A working group on radiation has been successfully organized to coordinate the 
important efforts in this area at the Jet Propulsion Laboratory and the Goddard Space Flight 
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Center. Other working groups cover parts data bases, Monolithic Microwave Integrated 
Circuits (MMIC), advanced microelectronics, and standardization. 

Vitro has also participated in and contributed to joint parts meetings with the ESA and the 
NASDA. Two meetings between ESA and NASA were hosted by Vitro, key Vitro 
participation was provided, and meeting minutes and action items were prepared and issued 
by Vitro. Also, Vitro personnel represented NASA Headquarters at an ESA Parts 
Conference in Noordwijk, Netherlands, presenting a paper on NASA activities and programs. 
Vitro also hosted two meetings between NASDA and NASA, providing key participation and 
coordination activities. 

A joint committee of participants from NASA and the AIA and NSIA industry associations 
was organized, with several meetings hosted by Vitro. Participation was provided by Vitro 
in an EEE parts working group. Procedures were set up so that NASA documents could be 
forwarded to the NSIA and AIA representatives to obtain coordinated industry comments on 
NASA documents and standards. 

Vitro also supported the participation of NASA representatives in a Space Parts Working 
Group that meets annually to provide interchange of parts information. The working group 
is jointly sponsored by NASA and the Air Force Space Division and includes participation by 
representatives other government agencies and from all major space industry contractors, 
subcontractors, and parts manufacturers. Organization of a NASA presentation for several 
meetings was organized and coordinated. 

Recently, a United States Space Parts Committee was organized as an outgrowth of the Space 
Parts Working Group to review and improve space parts practices, procedures, and activities. 
Vitro personnel have supported the committee at the request of NASA, providing members 
of several working groups and helping to present the NASA viewpoint in deliberations and 
documentation. 

Vitro personnel also provided support at meetings related to EEE parts sponsored by other 
NASA Headquarters offices and jointly prepared and presented papers with NASA personnel. 
These included a Space Transportation Conference at Williamsburg, VA, and a Memory 
Device Utilization Conference at Columbia, MD. Vitro representatives also periodically 
participated in meetings of the Space Station Parts Advisory Board and at meetings on parts 
problems related to the Space Shuttle, Mars Observer, the Hubble Space Telescope, and 
other projects. 

NASA Headquarters provides significant funding to field installations to have their 
contractors develop and implement research technical operating programs (RTOPs) to 
provide improvements in EEE parts reliability, standardization, and technology utilization. 
Vitro has significantly aided the NASA EEE parts manager in selecting, monitoring, and 
managing the development and implementation of such programs. 

A major RTOP involves parts standardization and data base activities at the Goddard Space 
Flight Center through a NASA Parts Project Office (NPPO). Vitro was instrumental in 
organizing meetings at Goddard at which the NASA personnel and their contractors who are 
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part of the NPPO provided presentations on the status of their activities. Also, support was 
provided in reviewing NPPO reports and activities and providing consultation to NASA 
Headquarters on effectiveness and recommendations for improvements and activities such as 
the issuance and updating of the NASA Standard Parts List MIL-STD-975 and comments to 
MIL parts specifications used by NASA. 

A number of RTOPs on advanced technologies are being conducted at the Jet Propulsion 
Laboratory. Vitro has accompanied NASA Headquarters personnel on numerous visits to 
JPL to review the status of the programs. Vitro recommended the initiation of an Advanced 
Microelectronic Parts Program to provide recommendations to NASA projects or reliability 
of devices in new technologies and recommendations as to their use. This program has 
resulted in a major addition to MIL-STD-975, with Vitro supporting its integration with the 
NPPO. Vitro also significantly contributed to the development of other JPL programs such 
as Radiation, Test Structures, an ASIC Manual, X-Ray inspection of metallization, and 
surface mount technology. 

Vitro has participated in yearly reviews of presentations by the RTOP managers and has 
provided recommendations for improvements and development of new programs. A database 
development program at the Lewis Research Center and a joint Microwave Monolithic 
Integrated Circuit program by Johnson Space Center and the Jet Propulsion laboratory are 
examples of successful programs which Vitro helped organize and put into effect. 

NASA Headquarters also funds the Naval Surface Weapons Center at Crane, Indiana, to 
provide EEE parts activities. Vitro has provided significant actions in monitoring the NSWC 
efforts and recommending new and improved support activities. Vitro personnel participated 
in a visit to Crane and participated in periodic meetings to review status and plans. Visits to 
the Air Force Rome Laboratory and the Defense Electronics Supply Center (DESC) also had 
significant participation by Vitro personnel. 

One of the initial tasks provided to NASA as part of the EEE parts program support was the 
preparation of a videotape on NASA EEE parts activities. Vitro personnel scripted the 
videotape and produced it, providing significant clips of EEE parts organizations and 
activities at NASA Headquarters, Goddard Space Flight Center, and Jet Propulsion 
Laboratory. The videotape was distributed to and presented at all NASA Field installations 
and to several Government agencies and contractors. The NASA EEE Parts Program 
videotape was praised by most viewers and is often used as an example of the type of v 
videotape other activities should prepare. It is periodically shown on NASA Select. 

Many technical presentations and papers were prepared for NASA parts personnel for 
meetings, conferences, and briefings. Major presentations and associated technical papers 
were prepared for the NASA Space Transportation Conference and the ESA Parts 
Conference. Vitro also prepared briefings for meetings of the NASA Parts Steering 
Committee, the Space Parts Working Group, and meetings with Government Agencies and 
Industry. Briefing charts were prepared for many other meetings and periodic reports to 
management on a continual basis, as often as every week. Vitro personnel developed, 
prepared, and presented briefings on a program to streamline the management EEE parts by 
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minimizing Nonstandard Parts Approval Requests and utilizing major space contractor parts 
management programs and parts lists. 

Vitro EEE parts personnel also prepared and presented monthly reports to NASA 
Headquarters personnel as part of the overall NASA support contract. These included 
monthly written status reports and monthly oral presentations. 

Vitro personnel participated on the EEE parts teams for the NASA Headquarters product 
assurance surveys of Field Installations. Support was provided in development of checklists, 
preparation of reports, and resolution of findings resulting from the surveys. Significant 
participation was provided at many of the survey sites, including Marshall Space Flight 
Center, Lewis Research Center, and Jet Propulsion Laboratory. 

A key aspect of the survey support was the synergy with other Vitro members of survey 
teams. Many of the findings were applicable to several areas, and Vitro efforts helped form 
a bridge between the areas. Also, the survey participation provided significant relationships 
for Vitro personnel with NASA personnel at the Field Installations. These relationships were 
important in providing support for many of the other tasks associated with the EEE parts 
program. 

Vitro personnel performed or participated in parts investigations and analyses, resulting in 
the preparation of reports with significant findings and recommendations. The efforts were 
related to general problem areas such as GIDEP alerts, Inspector General investigations, and 
data base development, as well as project-related investigations for the Space Shuttle, Mars 
Observer, Hubble Telescope, Space Station, etc. 

An important aspect of the EEE parts program task was related to GIDEP alerts. Continuous 
review of all GIDEP alerts on EEE parts was conducted, and monthly summaries of the 
alerts were prepared and distributed to Headquarters and Field Installation personnel. These 
summaries were included in parts newsletters which were distributed by the NASA Parts 
Project Office to NASA Field Installations and contractors. Also, many requests were 
received from NASA Headquarters personnel to perform specific searches of GIDEP alerts 
for specific parts and part manufacturers and to provide summary reports on the searches. 

Vitro personnel supported investigations and analyses related to Inspector General findings. 
We provided reports and consultative advice for investigations of fraud related to items such 
as resistors, circuit breakers, and semiconductor devices. Significant contributions were 
made to testimony provided to Congressional committees by the NASA associate 
administrator. Also, support was provided in meetings pertinent to the investigations with 
representatives of the Inspector General and other NASA Headquarters offices and Field 
Installations. 

Vitro supported many parts investigations related to specific projects. Vitro personnel 
participated in meetings on significant problem reports and provided recommended actions to 
teams formed to investigate specific problems. One effort was the review and discussions of 
proposals for the Space Station use of commercial parts and Class B parts. Inputs were 
provided to the team investigating hybrid microcircuit reliability in gyroscopes used on the 
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Hubble Telescope. Recommendations were provided in the investigation of possible sensitive 
switch failures related to Goddard Space Flight Center projects. Recent efforts were 
significant in supporting the Mars Observer investigation of possible failures of transistors, 
tantalum capacitors, and microwave devices. 

Another task Vitro performed was related to the development of databases for EEE parts. 
Reviews were made of the efforts and outputs of the NPPO development of the EEE Parts 
Information System (EPIMS), including participation in meetings and demonstrations. Also, 
Vitro personnel were instrumental in the development of a Folio-based data system which 
provides for the full-text search of reports from such sources as GIDEP alerts and the JPL 
radiation data bank. 

Specific general problem investigations were performed by Vitro personnel and reported on 
to personnel at NASA Headquarters and Field Installations. An example of this was a major 
investigation of failure modes of Philips resistors related to their use in NASA projects. 


m. CONCLUSIONS AND RECOMMENDATIONS 

NASA should consider having a single parts, materials, and processes program incorporating 
activities currently included in the Electrical, Electronic, and Electromechanical Parts; 
Mechanical Parts; Materials and Processes; Electronic Packaging; Non-Destructive 
Evaluation; and Metrology and Calibration Programs. 

As this contract closes, the Functional Management Self-Assessment Initiative is programmed 
to be implemented over the next few years. The Functional Management Program is very 
important to NASA. It ensures that suspected material weaknesses and problems are 
resolved and positive results are shared. Accomplishing the oversight functions improves or 
corrects each functional discipline or process. Code Q defined responsibility for Functional 
Managers at Headquarters and Field Installations and established the initiative at the 
Installation level. While continuing to pursue the initiative at the institution level, Code Q 
must emphasize the Headquarters program manager oversight assistance and involvement. 

There should be a uniform format for reporting self-assessment activity including results of 
both corrective actions and beneficial lessons learned, so that the individual Center reports 
may have recognizable commonality. Code Q should ask the Self-Assessment Working 
Group to define a common data base and oversee its creation, distribution, and operation to 
catalog and track self-assessment results at each installation. This will give the Center 
Directors and the Headquarters management a uniform view of the Center activity reports. 
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2000 SERIES TASK ORDER 
SAFETY AND RISK MANAGEMENT 


I. BACKGROUND FOR THE OVERALL TASK 

From the beginning of the contract, the impetus of the 2000 series task orders has been 
to provide oversight over the safety aspects of the Space Flight Programs and over the 
institutional safety programs throughout the 9 NASA Field Installations. A major part of 
those efforts included creation of new safety manuals and revision of existing ones. 

Typically, the 2000 series task orders provided four types of technical support to the NASA 
Headquarters Safety and Risk Management Division (formerly the Safety Division): 

• Analysis and assessment of selected NASA systems to ensure compliance with 
applicable safety standards — systems such as space nuclear power systems, the 
Space Station, the Space Shuttle, Unmanned Launch Vehicles, and payloads for both 
launch systems. 

• Safety assessments to ensure institutional compliance with safety standards, 
regulations, and legislated codes/consensus standards. The span of effort included 
safety of flight explosives and ground support operations unique to spaceflight, 
facility design and operation, occupational safety, fire protection, aviation safety, 
and hazardous materials handling. 

• Development of new policies, manuals, training courses, tools, and techniques. 
Included were such items as the basic safety policy for the Space Exploration 
Initiative; improved methods for performing Software Safety Analysis, Human 
Engineering Safety Analysis, and Risk Assessments; 3 computer-based systems for 
the NASA Safety Information System; automated support tools for creation and 
maintenance of Code QS safety documentation; 30 new safety manuals and NMIs; 
revisions to 16 existing safety manuals and NMIs; and 4 new safety training 
courses. 

• Establishment and maintenance of the Code QS technical library in a configuration 
that would permit quick access to safety reports and correspondence by Code QS 
personnel and to the databases for tracking problems and extracting pertinent data 
for evaluation by QS engineers. 


H. SUMMARY OF TASK ACTIVITY 

This task order has consistently comprised 16 distinct areas: 

Payloads System Safety Analysis* Aerosystems and Facilities 

Space Transportation System (Space Space Station (Freedom) Safety* 

Shuttle) Safety Analysis* INSRP (Nuclear Power System 

Safety) Support 
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NSIS (Safety Information Systems) 
Development 
Risk Assessment 
Software Safety* 

Human (Factors) Engineering* 
OSHA Safety and Fire Protection 
Flight and Ground Operations Safety 
(Explosives Handling Safety and 
Emergency Preparedness) 


Unmanned Launch Vehicle Safety 
and US System 

Safety Library Document and Safety 
Data Base (Records) Maintenance 
Aviation Safety 
Operational Engineering Panel 
Safety Documentation Development 
and Update 

Safety Training Development 


With the reorganization of Code Q in 1992, several safety areas (indicated with an asterisk) 
were deleted from the 2000 series of task orders, and others were added: Trend Analysis, 
System Assessment, Data Systems, and support for Prelaunch Assessment Reviews. _ 

Throughout the contract, Vitro engineers developed and maintained a keen awareness of 
critical issues impacting mission readiness in order to provide technically competent input for 
NASA decisions. By continual participation in Design Reviews, Phase Safety Reviews, 
Prelaunch Assessment Reviews, meetings of the Intercenter Aircraft Operations Panel, 
meetings of the DoD Explosive Safety Board, meetings of the Interagency Nuclear Safety 
Review Panel, etc., we were able to offer timely recommendations in real time as new issues 
surfaced. Our dedication to these tasks was instrumental in moving George Rodney/Q and 
Richard Perry/QP to confer a certificate of recognition for "Support to the Office of Safety 
and Mission Quality" in June 1990. We succeeded because of participation in the following 
reviews, teleconferences, and meetings: 

• Daily Space Shuttle teleconferences and meetings to stay abreast of Space Shuttle safety 
matters. 

• Daily unmanned launch vehicle tagup meetings to stay abreast of unmanned launch 
vehicle safety matters. 

• Payload Phase Safety Reviews to gain insight regarding the overall health of the 
payload safety process. 

• Facility Safety Reviews to gain insight regarding the overall health of safety processes 
for the wind tunnels and other facilities. 

• Space Station Preliminary Design Reviews to penetrate the complexities of and 
problems with its design safety program. 

• Meetings of the Interagency Nuclear Safety Review Panel to track the progress of the 
Program Offices’ assessments of radiological risks. 

• Meetings with NASA’s data base users to ascertain the real underlying requirements for 
each proposed NASA Safety Information System. 
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Headquarters surveys of the NASA Field Installations to determine their compliance 
with federally mandated safety requirements. 


• Project meetings at the NASA Field Installations that are developing new methods for 
Probabilistic Risk Assessment midwifing emerging risk assessment technologies. 

• Meetings with NASA’s software developers ascertaining the NASA environment and its 
potential impact on the adequacy of the software to provide safety. 

• Meetings with NASA Field Installation safety personnel working Human Engineering 
safety issues. 

• Meetings with NASA Field Installation safety personnel working suspended load crane 
safety issues, pressure vessel recertification issues, OSHA safety issues, etc. 

• Meetings of the National Fire Protection Association working the Halon alternatives 
issue (among others). 

• Meetings with NASA explosive safety officers working the issues of proper shipping 
and storage of chemical energy sources (hydrazine, ammonium perchlorate, and a 
variety of small solid rocket motors). 

• Meetings of the NASA aviation safety officers at all the field installations staying 
abreast of aviation safety issues in the NASA environment. 

• Various design reviews to obtain information to apply in technical assessments. 

• Trend Analysis Working Group meetings to gain insight to facets of the major programs 
that may benefit from trend analysis. 

• Meetings of the NASA Reliability and Maintainability Steering Committee to improve 
the NASA reliability program. 

• Meetings with NASA Field Installation safety personnel to gather an understanding of 
underlying issues and developing the appropriate wording for revising NHB 1700. 1(V- 
1), "The Basic Safety Manual." 

In addition, the Vitro safety team has completed an impressive number of projects during this 
contract period in the following key areas: 

Aerosystems and Facilities Safety. Provided updates to the NHB for Aerosystems Safety and 
the NHB for Facilities Safety, provided an update to the "Construction Safety" chapter of the 
Basic Safety Manual, participated in safety review meetings at three NASA Field Installations 
(ARC, LaRC, and LeRC), participated in wind tunnel project safety audits at these same 
three Installations, extracted failure data from the run records of the LaRC 8-foot wind 
tunnel in support of the probabilistic risk assessment performed by Code QS and provided a 
chapter for the final report explaining how the data was compiled, provided an evaluation 
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report on the DoD Facilities System Safety course, evaluated the safety aspects of wind 
tunnel rehabilitation and modification projects and provided comments to Lonnie Owen/QS, 
assessed NASA procedures and NASA Field Installation safety assurance programs for wind 
tunnel rehabilitation and modification projects, and participated in directors oversight 
committee meetings at the NASA Field Installations. 

Payload System Safety Analysis: Provided support for a total of 105 payloads; evaluated of 
269 payload safety data packages; developed and delivered to Code QS a working draft of 
NASA Safety Standard NSS/SY1740.X, "Payload and Cargo Element Safety Requirements"; 
and evaluated the Commercial Draft Specification of Standard Spacecraft/Launch Vehicle 
Interfaces for Medium and Intermediate Size Spacecraft. 

STS (Space Shuttle) Sqfety — Efforts in this area have been outstanding from the beginning. 
Our goal has always been to provide real-time Mission Safety Evaluations at each major 
decision point in the launch flow. The "Special Service Honor Award" conferred on the 
Vitro team in July 1990 attests to our success in that endeavor. 

We began by establishing a computer-based hazard tracking system. Then for 27 missions, 
we prepared and delivered up-to-the-minute Mission Safety Evaluation Reports immediately 
prior to each Prelaunch Assessment Review, each Flight Readiness Review, each Flight 
Readiness Firing, and each Launch-Minus-2-Day Review. Provided direct support to Jerry 
Moore/QS at KSC during each Flight Readiness Review, each Launch-Minus-2-Day Review, 
and each Launch-Minus- 1 -Day Review. Manned the Code Q Management Information 
Center for prelaunch support to Code QS at NASA Headquarters and to Jerry Moore/QS at 
KSC for problem assessment and resolution during the prelaunch hours for all 27 missions. 
Prepared and delivered the postflight editions of the Mission Safety Evaluation Report for 27 
Space Shuttle missions. Supported the System Safety Review Panel and Program 
Requirements Control Board Reviews for at least one session for each of the 27 Space 
Shuttle missions. Provided an update to NSTS 22254, "Methodology for Conduct of NSTS 
Hazard Analysis. " Developed a new initiative with Code QP and Vitro OSP personnel that 
proposed a forum for discussion and presentation of long-term Space Shuttle program 
enhancements. 

Space Station (Freedom) Safety — Began this effort by developing a Space Station Freedom 
Safety Program Plan, a Hazard Analysis Methodology Report, a Database Requirements 
Report, a first draft of SSP 30315, "Space Station Users Requirements Document," and a 
first draft of SSP 30309, "Safety Analysis and Risk Assessment Requirements." Provided an 
evaluation of 37 PDR data packages, and participated in 29 Space Station Freedom Design 
Review meetings for which synopsis reports were prepared for Code QS. Provided an 
evaluation of 31 Space Station Payload Safety Review Data Packages. Developed proposed 
NHB 1700.7C, "Safety Policy and Requirements for Payloads." Conducted comparison of 
FMEAs and Hazard Reports for the JEM Pressurized Module and Exposed Facility in 
support of the JEM PDR. Assisted in the definition of the basic safety policy for the Space 
Exploration Initiative. Provided an evaluation of the following documents: 

SSP 30XXX, "Space Station Freedom SSP 30599, "Safety Review Process 

Integrated Safety Program Plan" for Space Station Freedom Program" 
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NHB 5300.4 (X), "NASA Assurance 
Terms and Definitions" 

COL-RQ-ESA-027, "COLUMBUS 
Product Assurance/Safety Technical 
Requirements for Payloads To Be 
Integrated into the APM Launch 
Configuration" 

CR BB003198A/SSP 30652, "SSFP 
Safety Requirements for On-orbit 
Operations" 

NSIS Development : Performed an Independent Verification and Validation of the 
software for the Mishap Reporting/Coireetive Action System and the software for the 
Lessons Learned Information System. Developed a software tool to facilitate buildup of 
the Space Shuttle Mission Safety Evaluation Report. Developed a second software tool 
to facilitate maintaining an up-to-date Safety Training Catalog. Developed a third 
software tool, "The OSMA Document Management System Program," to facilitate 
incorporation of comments during the development or update of any OSMA document. 
Developed and provided a users guide for each of the three software tools. Worked on 
the NASA Code Q budget program software to restore the FY 1992 capabilities that 
were deleted when procedure was run to prepare the program to accept the FY 1993 
budget data. Participated in NASA Code Q Budget Program Joint Application 
Development meetings. Performed technical evaluation and requirements analysis for 
elements of the Decision Support System. Performed a technical evaluation of 
Relational Database Management Systems versus Text Management Systems for 
supporting Code QS information search and retrieval; performed a follow-up study 
providing additional detail on full-text, variable-field-text retrieval systems. Completed 
development of the Code Q Correspondence Management System concept specification, 
the draft Code Q Travel Tracking Program Specification, and the draft Code Q Local 
Area Network Management Plan. Developed the Lessons Learned Information System 
functional test check list used to control the preliminary verification testing of the 
INGRES version of the system. 

• Provided technical inputs to the system architectural design during in-process 
reviews held at the Assurance Data Systems Office, GSFC. Provided technical 
input to application development meetings on the Earth Observation System 
program development status monitoring, the Reliability Preferred Practices 
interface to Lessons Learned proposal assessment, the Potential Issue/Problem 
Identification Committee’s proposal for an Independent Assessment Catalog 
program, and the draft Fastener Information Management System design. 

• Assisted in bringing three systems through a complete development cycle - the 
Mechanical Parts Information Management System, the EEE Parts Information 
Management System, and the NASA Alerts Reporting System. For each of these 
three systems we evaluated the draft Requirements Specification, developed a set of 


The Revised SSF MB-06/Stage 6 
Integrated Fault Tree 

CR HH900106, "Safety Requirements 
Relative to Fail-Safe and Payload 
Failure Propagation" 


The Safety Analysis Report for the 
Crew Health Care System 
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test acceptance criteria to be used with the test procedure, prepared and distributed 
the Acceptance Test Traceability Report, worked with the PARAMAX in-house 
test team during the software functional verification that preceded the acceptance 
test, conducted the acceptance test and the acceptance demonstration, and prepared 
the Software Acceptance Test report. 

• Downloaded, assembled, and presented data used in Prelaunch Assessment 
Reviews. Wrote queries to retrieve detailed problem reports (when required) from 
the various NASA problem reporting databases, retrieved pictures from NASA 
Field Installations (when required) depicting damaged hardware or defects, and 
continuously acquired performance information on newer versions of the data 
retrieval equipment, seeking potential system upgrades. Maintained and updated 
the charts in the Code Q Management Information Center to reflect several types 
of data — trend analysis charts, technical issues, and generic trending. Drafted a 
document describing the entire Prelaunch Assessment Review process, including 
data creation, transmission, and display guidelines and requirements. Participated 
in a working group for automating the process of providing data support for the 
Prelaunch Assessment Reviews. 

• Provided an evaluation of the Assurance Data Systems Office Software 
Development Guide, the requirements for the EEE Parts Information Management 
System, and the draft Concept Specification for the NASA Assurance Information 
System. Conducted Customer Acceptance Testing for the EEE Parts Information 
Management System and prepared the test report. 

INSRP Support: Supported the development of the Safety Evaluation Report for both 
the Galileo and the Ulysses missions. Because these efforts had been forced into an 
extremely optimistic schedule, the production of these two reports to the White House 
required a great deal of close order drill in the final weeks. The degree of professional 
support provided by the Vitro team led to a letter of commendation from Chuck Mertz 
(then Director of the Safety Division) "for Outstanding Support to the Interagency 
Nuclear Safety Review Panel." 

In addition to providing continual suggestions for evaluating radiological risks, the Vitro 
team made a number of significant contributions. Provided the minutes for several 
meetings between the Interagency Nuclear Safety Review Panel and the Program Offices 
for the Galileo, Ulysses, Cassini, and TOPAZ Missions. Provided technical input to 
Launch Abort Subpanel meetings and Power System Subpanel meetings (in fact, the 
Power System Subpanel conferred recognition on the Vitro team via an 
acknowledgement in their subpanel report for the Ulysses mission, "For material 
contributions to several working meetings.") Completed an NMI on the use of 
radioactive materials in space and updated the chapter of the Basic Safety Manual 
dealing with the Interagency Nuclear Safety Review Panel and the process for evaluating 
radiological risks of missions that use nuclear power systems. Provided an evaluation of 
the following items: 
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The Final Safety Analysis Report for 
the Galileo Mission 

Several versions of the TITAN IV 
Databook developed by the Cassini 
Program Office 

The Radioisotope Thermoelectric 
Generator Safety Assessment for the 
Cassini Mission 

The Cassini Mission Environmental 
Impact Statement and its supporting 
studies 

Thr draft test report, "Feasibility — 

Investigation for Performing Fireball 
Temperature Tests" 

The Environmental Assessment for 
the Mars Environmental Survey 
Pathfinder Mission 

Risk Management : In this area. Vitro was a major player in assisting Code Q in 
introducing to the NASA Field Installations the newly emergent technology for 
performing risk assessments using statistical methods. Our efforts won a letter of 
recognition "For outstanding risk assessment support to the Risk Assessment Office" in 
August 1990 and a second such letter "For exceptional support to the probabilistic failure 
analysis assessment team" in January 1992. 

Assisted in the Code QS development of a workshop to disseminate newly evolved 
methodologies for Probabilistic Risk Assessment. Developed and executed a plan for 
enhancing the existing hazard analysis methods for the Space Shuttle. Provided extensive 
input into the INSRP uncertainty analysis for the evaluation process on the Ulysses 
mission. Evaluated several commercially available software programs for use in 
probabilistic risk assessment of system failure scenarios. Provided a position paper to 
Code QP on the required statistical precision of sample sizes to ensure an upper limit on 
uncertainty for their use during their NASA Field Installation surveys. Developed a 
prioritization scheme for Space Shuttle risks. Performed an evaluation of the JPL low- 
cycle fatigue estimation software code. Developed NMIs on future space debris 
limitations and on risk management requirements. Developed a position paper on 
reliability goals for the National Launch System. Developed a proposed revision to the 
Space Station Freedom risk assessment worksheet as presented in SSP 30309, "Safety 
Analysis and Risk Assessment Requirements." Developed a 3-year plan for activating 
the new risk assessment methodology throughout NASA. Provided an extensive review 
and rewrite of the Orbital Debris Handbook initiated by JSC. Coordinated the comment 
and rewrite effort for the Orbital Debris NMI and the Risk Management Policy NMI. 
Developed a list and rationale for future support projects in Risk Management/Risk 


The Final Safety Analysis Report for 
the Ulysses Mission 

The DOE Overall Safety Manual 

The TOPAZ II Preliminary Safety 
Assessment Report 

The Cassini Mission Earth Swingby 
Analysis 

The draft document, "Launch ~ 
Approval Process Guidelines" 
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Assessment. Developed a draft of a probabilistic design survey for MSFC and for the 
Engineering Management Council. Prepared a general outline for a Risk Assessment 
Handbook. Coordinated (with MSFC, LeRC, and Symbiotics Technologies Inc.) the 
evaluation of the Workstation (software, databases, and user’s guide) being developed 
under subcontract to Science Applications International Corporation. 


Participated in the joint effort on the uncertainty analysis for loss of berthable attitude 
for the Space Station redesign. In particular, developed the debris impact numerics and 
model and authored the Extra Vehicular Activity input portion of the report. Reviewed 
and modified the briefing and results for final presentation to the Space Station 
Transition Team. (The Space Station option assessment was performed in conjunction 
with Mike Frank/SFA, SAIC, and two members of Code QS -- Pete Rutledge and Ben 
Buchbinder.) Provided an evaluation of the following items: 


Solid Rocket Motor failure rate 
quantification refinement 

The TITAN IV Databook 

The JPL methodology for 
certification of Space Shuttle Main 
Engines 

The MSFC risk assessment method 
for the aero-assist Flight Experiment 

The ARC risk assessment for life 
support systems 

The JSC Space Shuttle assured 
availability program 

The JPL probabilistic failure 
estimation methodology 

A statistical analysis for Intelsat 
satellite regulator reliability 

An assembly risk assessment 
document for Space Station Freedom 

The response surface methods for 
MSFC 

An air pallet risk trade study for the 
Advanced Solid Rocket Motor 


A Range Safety Study performed by 
ACTA for the Air Force 

The FSAR uncertainty analysis 
generated by the Ulysses Program 

The MSFC WeiBayes method of 
extending risk assessment data 

The Code QP method for statistical 
analysis for air quality surveys 

The JSC integrated risk assessment 
plan for Space Station Freedom 

The MSFC single-flight-reliability 
estimation method 

The MSFC probabilistic methods for 
factor-of-safety calculations 

A FMEA/CIL ranking method 
developed by the University of 
Virginia 

A study on the Space Shuttle orbiter 
strut failure probability 

The reliability design goals for the 
National Launch System 

The JPL goodness-of-fit study for the 
failure data used in its method 


The system requirement design The KSC assessment of the JSC 

document for the National Launch Shuttle assured availability program 

System 
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Monthly University of Virginia 
reports on Risk Ranking and 
Screening 

The LaRC method for reliability 
trend assessment of small samples 

The JSC Orbital Debris assessment 
software 


The MSFC review of the JPL 
probabilistic failure assessment 
method 

A proposed JSC Management 
Instruction on Risk Management 

The JPL electro-optical probabilistic 
failure assessment proposal 


Unmanned Launch Vehicles : Performed an evaluation of the Orbital Maneuvering 
Vehicle software and provided comments to Lonnie Owen/QS. Initiated effort to 
generate a system safety handbook for expendable launch vehicles. Provided prelaunch 
planning support as well as support during the final launch countdown for a total of 11 
launches on a variety of unmanned vehicles. These efforts included the preparation and 
delivery of Mission Success Evaluation Reports immediately prior to each Flight 
Readiness Review as well as an update immediately prior to each launch. Completed 
preparation and delivery of seven postflight Mission Success Evaluation Reports. 
Participated in a number of preflight decision reviews: 


Delta/EUVE vehicle-on-stand 
review 

Mars Observer Project Launch 
Readiness Review 

Titan m/TOS preflight review 


NASA Headquarters Mission 
Readiness Review for 
Scout/SAMPEX 

Mars Observer Contamination red 
team follow-up review 

Titan III/TOS/Mars Observer 
Mission Readiness Review 


Titan III/TOS/Mars Observer 
mission director’s Flight Readiness 
Review 


Titan III/TOS/Mars Observer 
launch management Coordination 
Meeting 


Human (Factors) Engineering Safety Analysis : Reworked an existing Vitro methodology 
for performing human factors safety analysis and provided an Integrated Human 
Engineering/Safety Analysis Methodology for application to NASA development 
projects. Developed a draft Human Factors NMI, a preliminary NASA Handbook on 
Human Factors Safety, and a NASA Safety Standard, "Human Engineering Guidelines 
for Safety Assurance." Worked extensively with Code QS and the NASA Field 
Installations to initiate Human Factors Safety analysis efforts throughout NASA — a 
process that had been totally lacking prior to the onset of Vitro’s contract. These efforts 
included the development of quarterly human engineering briefings that were presented 
periodically to various NASA Field Installation personnel gatherings, the evaluation of 
the JSC Human Factors/Safety Training course, and the development of program plans 
for a prototype Human Engineering Program and a prototype Human Error Avoidance 
Demonstration Program. Also provided evaluation and oversight of the Human Error 
Avoidance Demonstration Program. Enhanced a video of the KSC student intern’s 
presentation of his NASA Human Factors Engineering evaluation. Also edited this 
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video (initially 1 hour and 15 minutes long) to create a 30-minute version and a 10- 
minute version for Code QS’ use in familiarizing NASA leaders with potential Human 
Factors program benefits. 

Software Safety : Reworked an existing (pre-NASA contract work) Vitro methodology for 
performing software safety analysis and provided a methodology for application to NASA 
software development projects. Worked extensively with Code QS and the NASA Field 
Installations to initiate software safety analysis efforts throughout NASA — a process that 
had been totally lacking prior to the onset of Vitro’s contract. These efforts included 
participation in software safety audits at the NASA Field Installations, evaluations of 
software development plans, participation in working group meetings, preparation and 
presentation of briefings and papers, and assistance in the development of safety 
requirements for software development design reviews, and participation in Technical 
Interface Meetings to establish a safety database for software. Converted an existing set 
of Vitro software safety objectives and provided a set of NASA specific software safety 
objectives. Assisted in numerous investigations of anomalous incidents throughout 
NASA systems that were found to be the direct result of software coding errors or 
software design oversights. Developed a NASA Software Safety Requirements 
Document. Developed a software safety standard and completed a software safety status 
report to be presented by Code QS to George Rodney/Q. Completed development of a 
NASA Software Safety Plan. Completed the draft NASA Software Safety Program plan 
and the draft NASA Software Safety Guidelines. Participated in Space Station Freedom 
software safety meetings and teleconferences to finalize software safety requirements and 
software documentation needs. Provided an evaluation of the following items: 

A Space Station Freedom Software The Software Systems Safety 

Fault Analysis Requirements Handbook prepared by JPL 

Document 

The Software Safety Program Plan 

prepared by JPL 

Aviation Safety. Worked extensively with Code QS and the NASA Field Installations to 
ensure adequacy of programs for aviation safety. These efforts included the development 
of an NMI on the subject of aviation safety, the development of a chapter on aviation 
safety for inclusion in the Basic Safety Manual, the development of a comprehensive 
aviation safety officers reference guide, the development of an aviation safety policy 
document for Code Q, assistance to Code J in the development of their aviation safety 
policy document, the development of a position paper on crew safety, research for Code 
QS in preparation for the proposed development of a kit for use by Space Shuttle 
mishap investigation teams, the development of a mishap investigation "fly-away" kit to 
help prepare investigators and other people involved in accident investigation, the 
Preliminary Draft Mishap Investigation Team Administration Manager Checklist of 
Duties, participation in the semi-annual meetings of the Intercenter Aircraft Operations 
Panel, participation in the periodic meetings of the Interagency Committee for Aviation 
Policy, participation in periodic aviation maintenance meetings, participation in periodic 
aviation safety reviews at the NASA Field Installations, participation as a team member 
during the NASA Intercenter Aircraft Operations Panel review of the aviation operations 
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at five NASA Field Installations (GSFC, MSFC, DFRF, LeRC, and KSC), and 
preparation of a paper that Vitro presented during the annual NASA Aviation Safety 
Officers Meeting. Participated in the ARC Aircraft Science and Applications Program 
Annual Review meeting. Provided an evaluation of the following items: 

Data on the F/ A 18 mishap NTSB mishap investigation checklists 

Federal Aviation Administration DoD mishap investigation-related 

mishap investigation checklists checklists 

OSHA and Fire Protection : Worked extensively with Code QS and the NASA Field 
Installations to ensure adequacy of programs for facility safety, occupational safety and 
health, fire protection, and hazardous material safety throughout NASA. Our effort won 
a NASA "Special Service Group Award" in August 1990, a letter of recognition "For 
professional support to the SRM&QA survey at LaRC' in February 1991, and a 
Certificate of Appreciation from the Construction Specification Institute "For outstanding 
achievements" in October 1991. 

The completion and distribution of NHB 1700.1 (Vl-B), "NASA Safety Policy and 
Requirements Document" was the most significant of these efforts because it will serve 
as the foundation for all safety programs at all nine NASA Field Installations ( and 
Headquarters) for the next decade. 

Efforts included developing a 5-year operating plan for electrical systems analysis; 
developing a 5-year operating plan for Fire Protection Systems Analysis; evaluating the 
fire risk NASA-wide; revising RP-1, "Recommended Practice for the Fire Protection of 
Essential Electronic Equipment"; participating in the periodic meetings of the Federal 
Fire Forum; tracking NASA compliance with the EPA regulations on HALON; tracking 
fire regulations at other Federal agencies; monitoring the fire protection systems at the 
NASA wind tunnel facilities, and revising the fire protection chapter of the Basic Safety 
Manual. Completed the NASA Safety Standard for Fire Protection. Maintained 
awareness of issues relating to Fire Protection requirements by participating in the 
National Fire Protection Association (NFPA) Annual Meetings, the NASA Fire 
Protection Engineers Meetings, the OSHA Electrical Safety Course presented via the 
NASA Video Teleconferencing System, etc. Also participated in the meetings of the 
NASA Hazardous Substances Internal Coordination Committee. 

Developed a new safety standard that was distributed to the NASA Field Installation 
Safety Directors under George Rodney’s signature (NASA/WS-1740.10, "NASA Safety 
Standard for Underwater Facilities and Non-Open Water Operations"). Participated as a 
member of the NASA Certification Review Board that inspected the Underwater Test 
Facility of McDonnell Douglas Space Station Division for compliance with NASA’s 
facility safety standards. 

Revised NMI 8621. IF, "Mishap Reporting and Investigation" and NMI 8710.2A, "NASA 
Safety and Health Programs." Participated in numerous NASA Field Installation 
inspection trips and surveys. Developed an NMI on the NASA Safety Awards Program, 
a revision to the NASA Safety Standard for Lifting Devices and Equipment (NSS/GO- 
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1740.9B), and a revision to the NMI establishing the requirements for a safety program 
for pressure vessels and pressurized systems (NMI 1710.3C). Participated in a 
collaborative effort with the OSHA to complete an OSHA approved NASA Alternate 
Safety Standard for Suspended Load Crane Operators. Developed a revision to the NMI 
on NASA Safety and Health Programs (NMI 87 10. A), a revision to the NASA form for 
reporting safety and health hazard abatement plans (Form 1584), a revision to the NASA 
form for reporting unsafe or unhealthful conditions (NASA Form 1390), and a revision 
to the instructions for NASA Form 1584. Developed a proposed NASA Form (and 
instructions) for employee reporting of alleged unsafe or unhealthful working conditions. 
Monitored OSHA compliance at the NASA Field Installations and at Headquarters and 
compiled the annual reports to OSHA. Prepared the Code QS input for NASA’s annual 
report to OSHA and periodically prepared tables showing NASA injuries/illnesses and 
mishap losses for Code QS. Planned and participated in a NASA Lifting Devices and 
Equipment Safety Conference and prepared the minutes. Completed NSS/GO-1740.9B, 
"NASA Safety Standard for Lifting Devices and Equipment." Compiled periodic change 
packages for the documentation on NASA suspended load operations for Code QS to 
transmit to OSHA. Prepared a draft document describing the Operational Safety 
Functional Management Review Process including checklists for use by the NASA Field 
Installations when performing self-assessments. Completed NMI 1740.3D, "NASA Safety 
Program for Pressure Vessels and Pressurized Systems." Compiled a "Headquarters 
Operational Safety Management Reference Book" to provide easy reference to elements 
of NASA’s Operational Safety Program to NASA safety personnel. Distributed the new 
OSHA Enforcement Directive, "Inspection Procedures for the Hazardous Waste 
Operations and Emergency Response Standard for 29 CFR 1910.120, Paragraph (q)" to 
all the NASA Operational Safety Managers. Maintained awareness of issues relating to 
OSHA requirements by participating in the OSHA 47th Annual Federal Safety and 
Health Conference, etc. 

Participated in a total of 10 Operations and Engineering Panel meetings for which Vitro 
provided the minutes and maintained the status on Requests for Action. Participated in 
a workshop with the Operations and Engineering Panel. Participated in the 8-Foot, 
High-Temperature Tunnel delta Integrated Systems Review at LaRC. Participated in 
the Director’s Oversight Committee meeting on the Unitary Wind Tunnel Modernization 
Project at ARC. Prepared an NMI to charter the Operations and Engineering Panel and 
coordinated the comments from the Code Q Division Directors and the SRM&QA 
Directors at the NASA Field Installations. 

Developed a revision to the NMI on the subject of the NASA Emergency Preparedness 
Program (NMI 1040.3), a NASA Headquarters Emergency Preparedness Plan, a training 
program for all NASA Emergency Preparedness Program coordinators, and checklists for 
use during surveys of the NASA Field Installations. Participated in NASA Headquarters 
meetings with the Federal Emergency Management Agency and prepared minutes. 
Revised NMI 1590.2, "Notification and Alert System for NASA Officials During Nonduty 
Hours," to include provisions for ensuring that NASA meets the new emergency 
communication requirements mandated by the White House. Developed a draft 
Standard Operations Procedures for NASA’s contribution to the Federal Emergency 
Management Agency (Aerial Reconnaissance) and distributed frequent Situation Reports 
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to all NASA Emergency Preparedness Coordinators regarding the 1993 Mississippi River 
floods for information purposes. Provided an evaluation of the following items: 

Numerous requests for safety-related The NASA mishap reports 

deviations or waivers 


The NASA Suspended Load 
Operation Analysis/ Approval 
Reports 

The OSHA Reform Bill 115 and 
related data 

The Headquarters Emergency 
Evacuation Plan 


KSCs procedure for handling 
approval of suspended load 
operations 

The NASA Field Installation safety 
program self-evaluation reports 

The NASA Safety Training Center’s 
new course on Fire Protection 


Flight and Ground Operations : Worked extensively with Code OS and the NASA Field 
Installations to ensure adequacy of programs for explosives facility safety and explosives 
handling procedures throughout NASA. Our dedication and professional skill won a 
letter of recognition from the chairman of the NASA Steering Committee of the Liquid 
Oxygen/Liquid Hydrogen Explosion Hazards Program in April 1990 "For contributions 
to the liquid oxygen/liquid hydrogen explosion hazards program" and a second such 
letter from Wayne Frazier/Code QS in December 1990 "For excellent explosives safety 
support." 

Efforts included the development of an NMI on the subject of obtaining shipping permits 
for rocket motors, the development of an NMI on the subject of explosive storage facility 
siting review and approval procedures, the development of an emergency exemption to 
the Department of Transportation to support the shipment of NASA explosive 
components, the development of a checklist for explosives storage and handling for use 
during NASA Field Installation surveys and audits, the development of strawman 
changes to 49 CFR 173, the development of NSS 1740.12, "Safety Standard for 
Explosives, Propellants, and Pyrotechnics," a revision to NSS 1740.14, "NASA Hydrogen 
Safety Handbook," a revision to NSS 1740.13, "NASA Oxygen Safety Handbook," 
research on an "expert system" (developed by ENSCO Corporation for the Joint Army 
Navy NASA Air Force (JANNAF) propulsion safety committee) to determine its 
applicability to NASA explosive classification systems, the development of a set of 
minimum test requirements for use in propellant development and test verification, 
participation (as the NASA representative) in the periodic meetings of the JANNAF 
explosives working group, participation (as the NASA representative) in the periodic 
meetings of the joint logistics chiefs, participation (as the NASA representative) in the 
periodic explosive hazard classification meetings of the Department of Defense Explosive 
Safety Board (DDESB), participation (as the NASA representative) in the periodic 
meetings of the Joint Ordnance Commanders Group (JOCG), participation in a number 
of NASA Field Installation surveys and audits, site checks of each ammonium 
perchlorate storage facility at the NASA Field Installations, and participation in the 
development of a solid rocket motor test plan for support of dual stacking operations in 
the Vertical Assembly Building at KSC. Assisted T. Mosikas at the Wallops Flight 
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Facility with procedures for storage and transportation of hydrazine for the Pegasus 
assembly and integration facility at the Facility. Participated in a Sounding Rocket and 
Balloon annual mishap review at the Wallops Flight Facility and a meeting concerning 
explosive proof wiring for working and storage facilities at the Facility. Assisted Wayne 
Frazier/QS, Jon Mullin/QS, and Code G with the NASA requirement to comply with 
the DoD Explosive Safety Manual or OSHA, researching 49 CFR Parts 100 to 177 
(October 1991 version) to determine NASA requirements for shipping rocket motors in a 
propulsive state. Submitted the NASA Explosive Safety Standards as a Supplemental 
Standard to CFR 1910.109 the OSHA Standard on Explosive Safety. Worked the issue 
of the housing of non-essential personnel in the KSC Vertical Assembly Building. 
Attended a Critical Design Review at LaRC on the Ignition Supply System in connection 
with the NASP Concept Demonstration Engine Model in the 8-Foot, High-Temperature 
Tunnel. Attended the Critical Design Review for the Liquid Hydrogen Structural Test 
Facility at DFRF. Provided an evaluation of the following items: 

NASA’s aerosystems pyrotechnic NASA’s liquid propellants operations 

devices 


NASA’s solid propellants operations 


The NASA-wide inventory of 
explosive components and their 
respective hazard classifications by 
DoD and DOT 

The Operational Safety Program 
developed by MFSC for the RSRM 
Propellant Safety Verification Test 

A paper prepared by Sandia 
National Laboratories on the DOE 
plan for explosive waste disposal 


The HEXDAM software program 
for siting new explosive storage 
facilities 

The siting criteria being used by 
each NASA Field Installation for 
determining placement of explosive 
storage facilities 

A study concerning development of 
hazards classification data on 
propellants and explosives 

The report by the Nevada 
Governor’s Safety Committee on the 
Pacific Engineering Production 
Company mfg site explosion in May 
1988 


The "blast wall" requirements for the 
mixer/casting bldg at the NASA 
Yellow Creek facility 

The Large Solid Rocket Motor 
Demilitarization Disposal Plan 
drawn up by the Joint Ordnance 
Commanders Group 

Data on an incident at the Morton 
International facility in Nevada 


The burn pad requirements for the 
NASA Yellow Creek facility 

The Chemical Process Safety Report 
publication 


The NASA Explosive Safety 
Orientation Course at JSC 
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Safety Library Document and Safety Data Base Maintenance: Worked extensively with 
Code QS and the NASA Field Installations to maintain (in a readily accessible system) 
the Safety Division’s safety database. Our efforts resulted in a letter of appreciation 
from Chuck Mertz/QS in July 1989 "For outstanding service." 

Efforts included periodic updating of the library, preparation of periodic bibliography 
reports, continuous additions to the database, periodic extracts of information for Code 
QS personnel both from the Code QS library and the libraries at each of the NASA 
Field Installations and at Headquarters, a revision to QS-DOI-9 1-001, "NASA 
Headquarters Mishap Notification Procedures," and two revisions to QS-DOI-9 1-002, 
"Directory of NASA Safety Personnel." Created budget tables for the Code Q Manager 
for Assurance Information Technology in preparation for her presentation to Safety 
Division Director. In support of the move to the new Headquarters Building, numerous 
boxes of records were transferred to the Federal Records Center; stamped 
CONFIDENTIAL and SECRET on over 60 classified documents when they were found 
to be improperly marked; contacted the NASA HQ Library to determine if they would 
retain the Mishap collection; reduced the size of the Technical Report Collection 
considerably by retaining only the front matter (i.e., cover, title page, table of contents, 
preface, and introduction) for reports that can be obtained from the NASA HQ Library, 
NASA Field Installations, or contractors; created a new file plan for each Q Code and 
distributed them to the appropriate personnel; pulled reports that had to be declassified 
or destroyed; created a new a list of classified materials; and investigated the possibility 
of transferring the Mission Safety Evaluation Reports to CASI/RECON, BWI to reduce 
the size of the collection and give the Mission Safety Evaluation report even greater 
visibility and distribution. Converted the format of a number of documents to permit 
revisions using the OSMA Document Management System developed by Vitro. (One 
document of note was converted for revision by the Joint Army Navy NASA Air Force 
explosive safety group — AFAL-TR-88-096 (Vols 1 & 2), "Space Propulsion Hazards 
Analysis Manual.") Created a compilation of documentation on Aviation Safety and 
distributed copies to Bill Comer/QS, Fred Gregory/Q, Tim McCarthy /JT, and the 
members of the Intercenter Aviation Operations Panel. Using the Mishap 
Reporting/ Corrective Action System, generated various safety reports for the Code Q 
Manager for Assurance Information Technology. Indexed, labeled, and filed materials in 
the Code QS Safety Software Library and created an inventory of materials, including 
the software name, version, numbers of copies, license numbers, and application. 

Trend Analysis : This subtask was part of the 2000 series task orders for only the last 15 
months of the contract. During that period of time, Vitro completed a draft 
document, "Qualification by Similarity Guidelines," that establishes uniform criteria for 
determining applications of this cost-saving approach to qualification. We also prepared 
a paper clarifying the application and definition of functional, hardware, and item 
criticality. Completed RP-1290, "NASA Trend Analysis Procedures" and performed an 
evaluation of the Space Station Freedom Trend Analysis Guide. Began developing a 
functional management review checklist/survey for determining if the NASA Field 
Installations are meeting SR&QA technical assessment requirements. The following 
investigations were made as part of the Trend Analysis effort: 
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The Secondary Seal Cavity Pressure 
Transducer Failure 

Space Shuttle Main Engine pressure 
sensor failures 

Window Cavity Conditioning System 
(WCCS) check valve failures 

Space Shuttle Main Engine open 
problem adverse trends 

Reliability and Maintainability : Participated in the January 1, 1993, Reliability and 
Maintainability Steering Committee meeting where we baselined eight new^ practices. 
Distributed meetings minutes and provided a letter announcing the next meeting to Code 
QS for distribution. Hosted and participated in the May 18-20, 1993, Reliability and 
Maintainability Steering Committee meeting where we baselined an additional 15 new 
practices. Distributed the initial printing of Supplement #2 of the Preferred Practices to 
the participating Field Installations along with requested copies of the original document 
and first supplement. Completed resolution of action items from the May meeting with 
NASA Field Installation members of the Committee. Conducted a meeting of the 
Systems Effectiveness Subcommittee on September 9 and 10 at Vitro to finalize the 
format of the draft, "Proven Design Techniques for Effective Maintenance Planning." 

The format of the document was finalized and presented the last week of September to 
the parent Committee. Minutes of the meeting and the update of the draft Strategic 
Plan were completed. 

• Provided course preparation materials to LeRC to support development of the 
NASA-wide Reliability Training course currently planned under Code Q funding. 

• Drafted a summary position paper for the NASA/Russian Space Agency panel for 
the Annual Reliability and Maintainability Symposium in January 1994 and 
distributed it for review by the Russians and Carl Schneider/QW. All comments 
were incorporated and the paper was submitted for publication in the symposium 
proceedings. Secured pledges for funding for the Russians’ transportation and 
lodging and reserved Aeroflot flights for their travel. Prepared a request for their 
visas to the U.S. Embassy in Moscow. Prepared and delivered support packets for 
use by Fred Gregory/Q at the symposium panels and the session he chaired. 
(ANSER Corporation acted as our interface with the Russians.) 

Technical Assessments: This subtask was part of the 2000 series task orders for only the 
last 15 months of the contract. One of the assessments performed by Ken Wong 
prompted Dr. Greenfield/QT to confer a letter of appreciation in May 1993 "For a 
commendable assessment report." 


Qualification by similarity 

The payload bay floodlight 

The misapplication of the hardware 
criticality field in the MSFC failure 
reporting databases 
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We revised NMI 8070.xx, "Verification of Space Flight Systems," based on comments 
received from reviewers at NASA Field Installations and Headquarters. Consolidated 
NASA-wide responses and review comments on the draft MIL-STD-1540C, "Test 
Requirements for Booster, Upper Stages, and Space Vehicle." Incorporated comments 
to the Lessons Learned Strategic Plan and Implementation Plan made by the NASA 
Field Installations. Wrote two chapters in the draft document "Dynamic Test Tailoring 
Guidelines." We also performed the following Independent Technical Assessments: 


Brake/Skid-Control Servovalve 
Jammed Open Failure Mode 


Aft Compartment Emergency 
Venting System 


Space Shuttle Orbiter Skin Corrosion Impact Damage to the Orbiter 

Windows 


The Use of Ammonia as the 
Working Fluid in the Space Station -- 
Freedom Thermal Control System 

Dynamic Testing Guidelines and 
Tradeoffs 


International Standardization of 
Environmental Testing for Space 
Programs 

Vibroacoustic Testing Guidelines 
and Tradeoffs 


Requirements for Structural Factors 
of Safety 


In addition to the above assessments, we also investigated a number of other anomalies: 


Shuttle Freon Coolant Loop Leakage 
Problems 


Shuttle Orbiter Vertical Tail 
Attachment Oversized Bolt-holes 


Oxidizer Preburner Augmented 
Spark Igniter Purge Check Valve 
Failure That Resulted in an On-pad 
Abort of STS-55 


Anomalies of the Pyro-valve Used 
on the CLUSTER Test Article by 
the European Space Agency 


Shuttle Orbiter Structural Corrosion Space Station Freedom Fracture 

Control Cost Reduction 
Considerations 


Shuttle Wing Leading Edge Pinhole 
Erosion 


Systems Safety Training Development : Developed course materials (including instructor 
guides, transparencies, and handouts) for four safety courses: 

Program Managers Safety Course Middle Managers Safety Course 

System Safety Managers Orientation Mishap Investigation Course 

Course 


The Middle Managers Course was presented to NASA personnel as a pilot course, and 
the Program Managers Safety Course continues to be presented as part of the overall 
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NASA training program for program managers. The final delivery of the Mishap 
Investigation Course training material was a first draft. 

Another significant contribution made by Vitro under this subtask was the development 
of the concept for a computer-based system to facilitate maintenance of an up-to-date 
Safety Training Catalog. (The development of the actual software tool for achieving this 
end was reported as part of the NASA Safety Information System subtask.) 

Sqfety Documentation Development and Update : The primary effort during this contract 
has been the updating of the Basic Safety Manual, now to be known as NHB 1700.1 (VI- 
B), "NASA Safety Policy and Requirements Document." As such, the manual was 
coordinated through a number of cycles of revision and distribution to the NASA Field 
Installations for review and comment. Our efforts in maintaining steady progress toward 
its completion inspired Wayne Frazier/QS to confer a letter of appreciation in April 
1991 "For outstanding support in documentation and training." 


Another significant contribution made by Vitro under this subtask was the development 
of the concept for a computer-based system to facilitate incorporation of comments 
during the development or update of any NASA safety document. (The development of 
the actual software tool for achieving this end was reported as part of the NASA Safety 
Information System subtask.) 


Under this subtask. Vitro also performed final production on 29 "first-time-ever" safety 
NMIs, handbooks, and standards that were developed as part of the technical efforts 
already noted in earlier paragraphs of this report: 

NASA Safety Standard A Range Safety Plan 

NSS/SY1740.X, "Payload and Cargo 
Element Safety Requirements" 

The final report of the Ad Hoc NSS 1740.11, "NASA Safety Standard 

Committee, "The Status Report of for Fire Protection" 

the STS Safety Risk Assessment of 

the Ad Hoc Committee," dated July 

1988 


A Hazard Analysis Methodology 
Report for Space Station Freedom 

A first draft of SSP 30315, "Space 
Station Users Requirements 
Document" 

Proposed NHB 1700.7C, "Safety 
Policy and Requirements for 
Payloads" (for Space Station 
Freedom) 


A Database Requirements Report 
for Space Station Freedom 

A first draft of SSP 30309, "Safety 
Analysis and Risk Assessment 
Requirements" (for Space Station 
Freedom) 

NMI 1700.8, "Space Debris 
Limitations" 
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An NMI on the subject of the use of 
radioactive materials in space 


A draft set of NASA specific 
software safety objectives 

A draft NASA Software Safety 
Standard 

Draft NMI 8070.xx, "Basic Policy and 
Procedures for Human Engineering" 

A NASA Safety Standard, "Human 
Engineering Guidelines for Safety 
Assurance" 

NASA Safety Standard NASA/WS- 
1740.10, "NASA Safety Standard for 
Underwater Facilities and Non-Open 
Water Operations" 

An alternative OSHA Safety 
Standard for KSC cranes 

An NMI on the subject of explosive 
storage facility siting review and 
approval procedures 

An NMI on the subject of aviation 
safety 

A draft document, "Qualification by 
Similarity Guidelines" 


A draft methodology for application 
of software safety analysis techniques 
to NASA software development 
projects 

A draft NASA Software Safety 
Requirements Document 

A Human Engineering/Safety 
Analysis Methodology 

Preliminary NHB 1700.1, Vol. 6, 
"Human Engineering Handbook for 
Safety Assurance" 

The Safety 2000 Document - 


The NASA Headquarters Emergency 
Preparedness Plan 


An NMI on the subject of obtaining 
shipping permits for rocket motors 

NSS 1740.12, "Safety Standard for 
Explosives, Propellants, and 
Pyrotechnics" 

An aviation safety officers guidebook 


This subtask also provided for the final production on Vitro-developed revisions to 17 
other safety NMIs, handbooks, and standards that were made as part of the technical 
efforts already noted in earlier paragraphs of this report: 


An NHB for Aerosystems Safety 

NSTS 22254, "Methodology for 
Conduct of NSTS Hazard Analysis" 

NMI 8710.2A, "NASA Safety and 
Health Programs" 

NMI 1710.3C, "Safety Program for 
Pressure Vessels and Pressurized 
Systems" 


An NHB for Facilities Safety 

NMI 8621. IF, "Mishap Reporting 
and Investigation" 

NMI 1040.3, "NASA Emergency 
Preparedness Program" 

NASA Safety Standard NSS/GO- 
1740.9B, "NASA Safety Standard for 
Lifting Devices and Equipment" 
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RP-1, "Recommended Practice for 
the Fire Protection of Essential 
Electronic Equipment" 

Volume 9 of NHB 1700.1, "Fire 
Protection" 

NSS 1740.13, "NASA Oxygen Safety 
Handbook" 

NMI 8070.4A, "Risk Management 
Policy for Manned Flight Programs" 

NMI 8070.XX, "Verification of Space 
Flight Systems" 

SUBCONTRACTS AND CONSULTANTS 


QS-DOI-9 1-001, "NASA 
Headquarters Mishap Notification 
Procedures" 

QS-DOI-91-002, "Directory of NASA 
Safety Personnel" 

NSS 1740.14, "NASA Hydrogen 
Safety Handbook" 

RP-1 290, "NASA Trend Analysis 
Procedures" (was NHB 8070.5) 


Vitro successfully elicited assistance from 1 1 subcontractors in the specialized safety area 
during the course of the Vitro contract to meet emerging NASA and OSMA 
requirements: 

Planning Research Corporation (PRC) 

Through a series of subcontracts with PRC, Vitro acquired for Code QS assistance in the 
Code Q independent risk assessment for the Galileo mission, an update for PRC’s 
computerized fault tree for potential Space Shuttle accidents, an integrated risk 
assessment software package for estimating the accident risks for the Ulysses mission, 
and a reliability estimate for the TITAN IV launch vehicle. 

BDM International, Inc. (BDM) 

Through a single subcontract with BDM, Vitro acquired for Code Q a conference paper 
on the Space Station risks associated with space debris, an associated bibliography, and a 
speech on the same subject for George Rodney. 

Lloyd Philipson 

Through a series of consulting agreements with Lloyd Philipson, Vitro acquired for Code 
QS assistance in the Code Q independent risk assessment for the Ulysses mission, 
refinements in the Space Shuttle accident probability model, and an evaluation of an Air 
Force Range Safety Analysis. 

Pickard, Lowe, and Garrick 

Through a series of subcontracts with Pickard, Lowe, and Garrick, Vitro acquired for 
Code QS an evaluation of a report on the Space Shuttle orbiter thermal protection 
system, "The Probabilistic Risk Analysis Model and Preliminary Observations." We also 
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acquired an evaluation of hazard assessment techniques currently used in the Space 
Shuttle program as opposed to emerging technologies for quantitative risk assessment. 
The final report was delivered on November 7, 1988, "Enhanced Hazard Analysis for 
Space Systems." 

The University of Virginia 

Through a series of subcontracts with the Center for Risk Management of Engineering 
Systems at The University of Virginia, Vitro acquired access to two new risk assessment 
technologies for Code QS -- the Partitioned Multiobjective Risk Method for Extreme 
Value Events and the Evaluation of Identification and Ranking Methods for FMEA/CIL 
Items. 

Safety Factor Associates (SFA) 

Through a series of consulting agreements with SFA, Vitro acquired for Code QS 
enhancements in the Space Shuttle accident probability model, major contributions to 
the joint Code Q/M reliability model for the Space Shuttle, assistance in the Code Q 
evaluation of the MSFC Assured Shuttle Availability model, several workshops for 
NASA personnel to become proficient in the new techniques for probabilistic risk 
assessment, assistance in the development of the appropriate failure estimates by the 
Failure Probability Splinter Group for use in the TITAN IV launch vehicle Databook for 
the Cassini Mission, an "Uncertainty Analysis" that was the basic source for the error 
bars placed on the final results of the nuclear safety evaluation performed by the 
Interagency Nuclear Safety Review Panel in their report to the White House on the 
radiological risks of the Ulysses mission, assistance in the Code Q independent risk 
assessments for both the Galileo and Ulysses missions, two Bayesian analysis software 
modules for use in Probabilistic Risk Assessment, assistance in an integrated risk 
assessment of the Micro-rover for the Mars Environmental Survey mission, assistance to 
the Radioisotope Thermoelectric Generator Working Group, assistance to the 
Interagency Nuclear Safety Review Panel for the Cassini mission, and a calculational 
model for the Space Station assembly integrated risk assessment. Provided a report 
entitled "Model for Uncertainty Analysis of Frequency of Failure to Attain HTC for 
Space Station Transition Options A-l and A-2." Also provided a diskette with the model 
described above. (The Space Station option assessment was performed in conjunction 
with SAIC, Bob Weinstock/Vitro, and two members of Code QS — Pete Rutledge and 
Ben Buchbinder.) 

Technical Analysis, Inc. (TAI) 

Through two separate subcontracts with TAI, Vitro acquired for Code QS an Ordnance 
Siting and Certification Plan and the expert services of Jim Wiggins, who participated as 
a member of the Ad Hoc Committee on its 1988 evaluation of the Space Shuttle risk 
management system. The Ad Hoc Committee developed a final report, "The Status 
Report of the STS Safety Risk Assessment of the Ad Hoc Committee," dated July 1988. 
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Operations Research, Inc. (ORI) 

Through a single subcontract with ORI, Vitro acquired for Code Q the expert services of 
Joyce McDevitt, who participated as a member of the Ad Hoc Committee for its 1988 
evaluation of the Space Shuttle risk management system. The Committee developed a 
final report, "The Status Report of the STS Safety Risk Assessment of the Ad Hoc 
Committee," dated July 1988. 

Louis J. Polaski 

Through a single consulting agreement with Mr. Polaski, Vitro acquired for Code Q the 
expert services of Louis J. Polaski, who participated as a member of the Ad Hoc 
Committee for its 1988 evaluation of the Space Shuttle risk management system. The 
Committee developed a final report, "The Status Report of the STS Safety Risk 
Assessment of the Ad Hoc Committee," dated July 1988. 

Symbiotic Technologies, Inc. (STI) 

Through a single subcontract with STI, Vitro acquired for Code Q a survey of 
probabilistic design methods and the initiation of an evaluation of the commercially 
available PROBAN software program and NESIS, a software program developed on a 
LeRC contract. Both software programs are for use in performing probabilistic 
structural analysis. 

Science Applications International Corporation (SAIC) 

Through a single subcontract with SAIC, Vitro acquired for Code Q a series of seminars 
on risk assessment and risk management (along with tutorial materials). Seminars were 
given at NASA Headquarters, at JSC, and at LeRC. We also acquired an evaluation of 
three candidate launch vehicle architectures proposed by MSFC for the advanced space 
transportation development program — a "Shuttle forever" architecture, an integrated 
cargo transfer return vehicle/personnel launch system architecture relying on an 
expendable launch vehicle with 50,000-pound launch capacity to low Earth orbit using 
Saturn F1A and J2 engines, and a medium cargo transfer return vehicle architecture in 
which a 90,000-pound-capacity vehicle is derived from the 50,000-pound core. 

We also acquired a PC-based Excel™ software module for generation and analysis of 
event trees and consequence trees (part of the prototype Space Systems Reliability Data 
workstation NASA acquired under another SAIC subcontract through the Air Force). 
Under the Vitro subcontract with SAIC, we procured several demonstrations and trial 
uses of the entire workstation for NASA Headquarters managers and Vitro personnel. 
We also procured a User’s Guide for the entire Workstation. 

Participated in a top-level assessment of the two most viable options for the proposed 
Space Station redesign to ascertain the relative programmatic risks. (The Space Station 
option assessment was performed in conjunction with Mike Frank/SFA, Bob 
Weinstock/Vitro, and two members of Code QS — Pete Rutledge and Ben Buchbinder. 
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III. CONCLUSIONS AND RECOMMENDATIONS 


The Vitro Safety team has grown to understand the variety of difficulties that hinder the 
Code QS efforts to bring real improvements to NASA Safety Programs. Our efforts over 
the past 6 years have been devoted to improving the ability of the program managers to 
make informed decisions regarding safety and to improve the handbooks and manuals 
used throughout NASA on a daily basis. The following recommendations were derived 
from our comprehensive analysis of NASA safety programs: 

• During our many visits to the NASA Field Installations on safety surveys and audits, 
it became clear that one Field Installation (LaRC) enjoyed far better support for 
the safety programs from the "Program" side of the house. We suspect the 
excellence of LaRC’s safety program is due to a single difference in their 
management approach. The SRM&QA Director at LaRC has a 20% input to the 
annual performance appraisal for each of the other department heads. 

OSMA may wish to have the NASA SRM&QA community consider allowing each 
of the SRM&QA Directors a 20% input to the annual performance appraisals for 
their respective department heads. Additionally, Code Q may wish to suggest a 
similar Code Q input into SRM&QA Directors evaluations. 

• One of the most significant contributions Vitro made to the Safety and Risk 
Management effort was to advance our expertise in software safety. A great deal 
has been accomplished to introduce the new idea within the NASA Programs that 
software has become intensively involved in systems designs and can no longer be 
treated in the old, familiar manner. However, perhaps it is time to step-up the 
program. NASA should consider seeking out an experienced software safety person 
to add to the Code QS staff. NASA Code QS could also benefit from the 
experience of an on-staff Fire Protection expert. 

• In the 6 years of this contract, we have witnessed wide fluctuations in Code QS’s 
travel budget. The reduced number of trips to the NASA Field Installations in the 
lean years severely hampered Code QS’s ability to perform its function. A great 
deal of interaction is really needed, if we are to understand the problems faced by 
the safety personnel in the field. Consider giving the travel budget higher priority 
in the future. 

• Safety is most effective in the years before CDR. This holds true for the Space 
Station as well. We cannot help feeling that insufficient attention is being given to 
that effort, and soon it will be too expensive to correct any safety problems in the 
design. Consider applying a larger portion of the Code QS (or Code Q) effort to 
the oversight of the Space Station safety program. 

• Our Explosive Safety expert is already on record regarding the blast wall that is in 
the plans for the mixer/casting building at the Yellow Creek Facility in Iuka, MS. 
Although funding has been temporarily eliminate that facility, the blast wall remains 
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in the design. Our expert feels the $3 million could better be spent elsewhere 
because OSHA does not require such a wall and because there is no real evidence 
that the wall could protect the workers on the "safe" side. 
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3000 SERIES TASK ORDER 
PAYLOADS AND AERONAUTICS 


I. BACKGROUND FOR THE OVERALL TASK 

In the beginning of the contract, three task areas (systems assessment, trend analysis, and 
data systems) were formed to assess and communicate to top NASA management safety, 
reliability, maintainability, and quality assurance (SRM&QA) problems that could impact 
mission success. Because these three functions are synergistic activities requiring a 
coordinated effort, the Systems Assessment and Trend Analysis Division (Code QT) was 
formed in 1990 to consolidate these efforts. Vitro support to the new Code QT was 
provided under the 3000 series task order. Prior to 1990, the Data Systems/Trend 
Analysis and Systems Assessment were two separate divisions where Vitro support was 
provided under 3000 and 4000 series task orders, respectively. 

In support of Code QT, the Vitro tasks encompassed the following: 

• Data Systems'. Evaluate, develop, and implement NASA SRM&QA data 
base/information management systems. Manage, operate, and maintain a 
headquarters-level SRM&QA management information center (MIC). Provide data 
systems support for the Space Shuttle prelaunch assessment review (PAR) process. 
Operate and utilize problem reporting and corrective action systems to write 
queries/extract data to support engineers conducting trend analyses and technical 
assessments. 

• Trend Analysis: Develop, implement, and assess NASA trend analysis programs at 
Headquarters, Field Installations, and contractor sites. Develop, revise, and update 
documentation (management instructions, handbooks, standards) for use as 
Agencywide guidelines and procedures for performing trend analysis. Perform 
trend/correlation analyses to identify adverse trends and anticipate potential 
problems. 

• Systems Assessment : Conduct independent technical engineering assessments at the 
system, program, and element level. Provide an independent "second look" at 
engineering issues/concerns in the SRM&QA areas for Code Q. Participate in and 
provide technical backup for design reviews, flight readiness reviews, and test 
readiness reviews. Prepare an assessment report that includes problem description, 
information collected, analysis results, conclusions, and recommendations for 
resolution of problem areas. 

In December 1992, a Code Q reorganization resulted in a reassignment and revision of 
task orders. The systems assessment, data systems, and trend analysis functions were 
transferred to the Safety and Risk Management Division (Code QS) under the 2000 
series task order. Code QT was renamed as the "Quality Management (Payloads) 
Division." This new organization was established as a direct interface with the Payload 
Program Offices for all OSMA functions. With an Agencywide goal of conducting all 
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missions "better, faster, cheaper," the new Division was created to ensure that limited 
SRM&QA resources were utilized more effectively throughout the life cycle of a payload 
development program. Particular emphasis was directed towards a more active 
involvement of Code Q in the early phases of the program. 

In January 1994, Code QT was again renamed /changed to the "Payloads and Aeronautics 
Division.” The roles and responsibilities of this new division have not yet been fully 
defined, but will encompass SRM&QA activities for both Expendable Launch Vehicle 
(ELV) and Aeronautics programs in addition to the continuing support for NASA 
Payload programs. 


II. SUMMARY OF TASK ACTIVITY 

DATA SYSTEMS/TREND ANALYSIS/SYSTEMS ASSESSMENT 
Data Systems 

Vitro increased the use of information technology in support of Code Q meetings. Vitro 
was heavily involved in the design of a system to improve the overall effectiveness of the 
PAR process through expanded use of automation. Vitro evaluated numerous software 
and hardware packages for potential use in automating the PAR process. PAR 
automation tools using NeXT workstations, various software packages, and peripherals 
were integrated by Vitro. Vitro coordinated the installation of equipment for still-image 
capture and transmission and installation of the BARCO projection television in the 
Code Q Management Information Center (QMIC) to support meetings and 
presentations. 

Vitro staff were extremely responsive in writing queries to extract data from the Program 
Compliance Assurance and Status System (PCASS)/Problem Reporting and Corrective 
Action (PRACA) systems to support technical assessments being conducted by both 
NASA and Vitro engineers. The Vitro data systems staff, as experts in PCASS/PRACA 
operation and utilization, provided training to numerous Code Q, Code M, and Vitro 
engineers. Vitro identified data integrity problems (e.g., missing, incomplete, or 
incorrect codes/data) within the PCASS/PRACA data bases. Subsequently, a major 
effort was undertaken by the NASA Field Installations to improve data accuracy and 
completeness within their PCASS/PRACA data bases. 

Vitro maintained, managed, and operated a SRM&QA QMIC to display correlated 
technical and management data to identify trends and potential programmatic impacts. 
For each launch, a briefing book executive summary containing QMIC charts was 
prepared for the Code Q Associate Administrator. Vitro assisted in the development 
and implementation of a significant problem reporting system to track/monitor 
significant problems and bring them to the immediate attention of top NASA 
management. This system was helpful in tracking the status and disposition of several 
hundred significant problems that had to be resolved prior to the Space Shuttle return to 
flight. 
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Trend Analysis 

Vitro assisted NASA in developing and updating the following trend analysis 
documentation: 

• NMI 8070.3, "Problem Reporting, Corrective Action, And Trend Analysis 
Requirements" (establishes the NASA policy and requirements for the conduct and 
reporting of trend analysis). 

• NASA Reference Publication (RP-1290), "NASA Trend Analysis Procedures" 
(provides uniform guidelines for implementing and conducting trend analyses for 
aeronautics and space programs). 

• NASA-STD-8070.5, "Trend Analysis Techniques" (describes mathematical/statistical 
techniques). 

In addition, Vitro authored a Space Station Trend Analysis Guide. This document 
identifies trend analysis documents, activities, projects, and resources that are available 
and applicable to the Space Station. This guide was distributed to the NASA Field 
Installations for review and received favorable comments. 

Vitro played a key role in the inception and development of the NASA Trend Analysis 
Working Group (TAWG), which provided a forum or mechanism for developing and 
disseminating state-of-the-art trend analysis knowledge, tools (statistical software), and 
techniques. Vitro prepared trend analysis presentations, meeting minutes, and agendas 
and reviewed trend analysis proposals, projects, and activities. Support for the TAWG 
meetings resulted in a significant increase in trend analysis projects and activities. 

Vitro conducted many trend analyses to assess or identify recurring problems. For 
example, Vitro assessments concerning failure trends and state-of-the-art technology for 
spacecraft data recorders, gyros, and deployable mechanisms /structures were prepared 
and commended by upper Code Q management. 

Technical Assessments 

Vitro assessed numerous technical and critical issues associated with the Orbiter, Solid 
Rocket Motor, Solid Rocket Booster (SRB), Space Shuttle Main Engine and External 
Tank, Space Station, payloads, and specific engineering issues (e.g., spacecraft testing). 
(Note: Vitro completed more than 180 technical assessments. At the end of each fiscal 
year, selected assessments were bound and provided to Code QT for future reference 
and archival purposes.) Technical Interchange Meetings were held periodically in which 
Vitro personnel provided briefings of interim and final results of current technical 
assessments to upper NASA management including the Code Q Associate Administrator. 

Several letters of commendation were received by Vitro personnel for outstanding 
performance or accomplishment of urgent and critical assessments. Two Vitro 
employees who supported Code QT were honored with the prestigious Manned Flight 
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Awareness (MFA) award. Vitro engineers were also selected for monitoring and 
participating in special investigation teams and technical committees that included the 
following: Task Force 10 for Geostationary Operational Environmental Satellite 
(GOES), Galileo High Gain Antenna (HGA), Advanced Technology Satellite (ACTS) 
Transfer Orbit Stage (TOS), HST Gyro Failure Review Board, and Space Station 
Electrical Grounding Tiger Team. 

Many of the technical assessments received high visibility or contributed to the decision- 
making process. For example, during the STS-37 prelaunch assessment of the cracks in 
the hinge of the ET doors, a Vitro stress analysis indicated negative margins of safety. 
This analysis/assessment, which addressed launch mission risk, provided Code Q 
management rationale for delaying the launch until the problem was corrected. Other 
critical Shuttle-related topics assessed by Vitro included software, instrumentation 
(transducer crack), and SRB aft skirt factor of safety issues. 

Vitro evaluated and analyzed the Shuttle avionics systems (software/hardware) issues 
and performed independent assessments of problems and recommended corrective 
actions. In support of Space Shuttle flight readiness reviews, Vitro provided Code Q 
with a software readiness package/assessment prior to each launch. These assessments 
provided Code Q with rationale for accepting the Certification of Flight Readiness. 
Furthermore, Vitro maintained and updated a flight software manifest related to the 
Shuttle flight schedule and significant software changes. 

Highly complex and technical Space Station issues were identified by Vitro and brought 
to the attention of the Space Station Deputy Director. Vitro identified and assessed 
critical Electrical Power System (EPS) stability and plasma contactor issues and 
concerns. Vitro identified the need for fully reviewing the EPS design and the need for 
IV&V analysis and enhanced systems testing. Vitro worked with Level II (LeRC) in 
developing an EPS secondary stability change request (CR) that would develop criteria 
to ensure system-level stability. EPS stability issues raised were instrumental in the 
formation of an Engineering Design Council tiger team. In addition, a Vitro assessment 
(Low Earth Orbit Plasma/Space Station Electrical Power System) led to a series of 
investigations that resulted in the procurement/development of a plasma contactor. 


QUALITY MANAGEMENT (PAYLOADS) 

Vitro conducted a study on vibroacoustic testing risks and tradeoffs. A technical paper 
was prepared based on this study and was presented at the Spacecraft Launch Vehicle 
Dynamics Environments Technical Interchange Meeting as well as at the ESA/ESTEC 
Environmental Testing for Space Programs Symposium. Vitro is preparing a dynamic 
test tailoring guidelines document as a follow-on to the vibroacoustic study. Vitro 
personnel are members of special aerospace advisory panels for professional 
organizations such as the Aerospace Testing Seminar Advisory Panel. 

Vitro conducted an assessment that surveyed and evaluated NASA, U.S. Air Force/DoD, 
and aerospace industry guidelines and practices on the use of qualification by similarity 
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to reduce testing of space flight hardware. Because the assessment identified differences 
in content and level of detail guiding use of qualification by similarity among these 
organizations, Vitro developed and proposed uniform NASA-wide guidelines on 
qualification by similarity. 

As part of NASA’s goal of reducing the cost of scientific space missions. Code QT 
conducted a workshop aimed at reducing the cost of quality. Personnel from each Field 
Installation, major NASA contractors, NASA Headquarters, and academia participated 
in this 3-day workshop at the University of Maryland. Vitro was an active participant in 
the development and coordination of this highly successful workshop. The success of this 
initial effort has led to a continuing initiative at each of the payload Field Installations. 

Vitro assisted in revising NASA Management Instruction (NMI) 8010.1, "Classification of 
NASA Payloads." The intent of the revision was to provide payload developers increased 
flexibility Tn determining how best to maximize existing SRM&QA resources while still 
maintaining an acceptable level of risk. The revised NMI is still in draft form, but will 
continue to be a necessary component in implementing an effective payload SRM&QA 
program. 

In response to a request by the Code S Program Office, Vitro assisted Code QT in 
evaluating the impact to the overall reliability of the Earth Observing System if EEE 
parts requirements were reduced. Vitro provided the results of this assessment to the 
Program Office and assisted them in the decision-making process. 

At the request of the Submillimeter Wave Astronomy Satellite (SWAS) Program 
Manager, Vitro assisted Code QT in conducting a reaction wheel study. The purpose of 
the study was to evaluate a Project decision to change the proposed design and to 
eliminate spare components. The results of the study presented to the Program Manager 
indicated the Project was proceeding in an appropriate manner. 


III. CONCLUSIONS AND RECOMMENDATIONS 

Vitro has been very responsive in meeting the changing requirements of Code QT over 
the life of the contract. Despite numerous reorganizations, downsizing, and restructuring 
of tasks, Vitro has effectively realigned its staff to provide the right skill mix to meet the 
needs of Code QT. Vitro has contributed significantly to the areas of data systems, trend 
analysis, and systems assessment. The data integrity of the PCASS/PRACA systems has 
improved significantly. This has facilitated and enabled performance of more accurate 
trending. The development of NASA trend analysis documentation and establishment of 
the TAWG has significantly increased trend analysis activities and projects throughout 
NASA. Many Space Shuttle, Space Station, and payload-related systems/technical 
assessments prepared by Vitro received high visibility and have contributed to the 
SRM&QA decision-making process. 

The efforts of Code QT and Vitro have gained increased visibility in the Program Offices 
under the new Quality Management (Payloads) Division. This new visibility has allowed 
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the Code QT/Vitro team to become a value-added contributor to the development of 
NASA payload programs. 

The new direction the Payloads and Aeronautics Division is taking must ensure OSMA 
continues to increase its involvement in the early phases of the Payload Development 
programs. This increased visibility and participation in the early decision-making process 
will be required if NASA is to maintain an acceptable level of risk with increasingly 
limited resources. The OSMA should be actively involved in payload program activities 
throughout the duration of the program (from early development through integration and 
launch). Also, Code QT needs to maintain the SRM&QA function as an aggressive 
contributing element in the planning, development, and implementation of the Payload, 
ELV, Upper Stages, and Aeronautics programs. NASA should maintain a process that 
clearly defines, evaluates, and articulates the impact of program decisions on mission risk 
throughout program life cycles. This process should include oversight, independent 
assessment, performance of contract (RFP and SOW) reviews, and active participation as 
members of Program Design Review Boards, Change Boards, System Effectiveness 
Review Boards, Flight Readiness Review Boards, and Launch Management Teams. 
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4000 SERIES TASK ORDER 
ENGINEERING 


I. BACKGROUND FOR THE OVERALL TASK 

OSMA develops and advances Agencywide engineering standards and practices that form 
the backbone of NASA’s safety and mission assurance capability. OSMA’s programs in 
software engineering and assurance, applied technology, systems engineering, and metric 
transition contribute directly to engineering standards. OSMA also assesses flight 
readiness of Shuttle software. OSMA’s systems engineering and metric transition efforts 
are integrated with national initiatives (involving other government agencies) to improve 
the competitiveness of U.S. industry in international markets. 

This task began in January 1991, shortly after the Technical Standards Division’s 
organization, and included efforts in engineering standards, metric transition, and 
advanced technologies. Task 1000 previously incorporated the engineering standards and 
metric transition activities. In 1993, the scope of the task order expanded to include 
software engineering and assurance, electronic part packaging, and EEE parts, when 
these technical areas became responsibilities of the newly named Engineering Division. 
Earlier these activities were supported by Tasks 1000 and 3000. 


II. SUMMARY OF TASK ACTIVITY 

For all engineering programs, Vitro’s proactive engineers developed sensible plans and 
independently reviewed work done by the Field Installations and outside organizations. 
We developed new OSMA procedures for expediting the preparation of technical 
documents. We assisted with the organization of and participated in high-level NASA 
steering committees and workshops. Vitro’s staff met with the OSMA staff to plan 
activities and worked as a team to provide responsive, timely, and consistent support. 

Our accomplishments were documented in Monthly Technical Progress Reports, informal 
monthly highlighted briefings, and specific technical reports. 

TECHNICAL STANDARDS 

To improve OSMA’s processes for developing, revising, adopting, and tailoring standards, 
Vitro helped Code QW develop the NASA Standardization Procedures Handbook (NHB 
8070.XX). The new development process is superior to current approaches because: 

• The Agencywide NMI/NHB process is more difficult and time-consuming and is not 
specifically designed for standards. 

• The more direct Field Installation processes do not yield uniform standards. 

While NHB 8070.XX is not yet ready for the final approval and concurrence process, we 
used it on a trial basis for some software-related documents. 
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METRIC TRANSITION 


Public Law 100-418 and Executive Order 12770 require all Federal agencies to adopt the 
metric system of measurement. Compliance is important because this national initiative 
brings higher productivity and international competitiveness to U.S. industry. OSMA 
leads an Agencywide metrication effort and maintains high-visibility contacts with the 
Congress, the Department of Commerce, and other Federal agencies. OSMA, with help 
from Vitro, established the NASA Metrication Planning Group (NMPG) at 
Headquarters to plan the Agency’s metric transition and coordinate its implementation. 
Using Vitro’s sound advice, OSMA modified NASA’s metric use policy, prepared a 
metric transition plan, and compiled annual reports to the Congress. 

The Administrator approved NMI 8010.2A, "Use of the Metric System of Measurement 
in NASA Programs," in June 1991 and NASA’s Metric Transition Plan including a Metric 
Waiver Process in February 1992. The policy made OSMA responsible for "establishing 
procedures to implement this policy"; the plan assigned three activities to OSMA: overall 
leadership of NASA transition activities, administering the plan’s waiver process, and 
identifying requirements for metric standards. The NMI was an update of the earlier 
policy that incorporated changes required to comply with Public Law 100-418 and 
Executive Order 12770. Vitro reviewed plans and reports prepared by individual 
program offices and Field Installations, looking for sound engineering practices and 
consistency among programs, Field Installations, and Federal agencies, and identified 
items for OSMA that required clarification or improvement. When OSMA received a 
waiver request, we evaluated it using the same criteria and suggested activities to OSMA 
that the development program could perform to expedite NASA’s metric transition. 

Vitro assisted OSMA with preparation of the annual metrication report to the Congress. 
This report, submitted to the Congress in January, covers progress in the past year and 
outlines activities planned for the current year. In addition, we helped OSMA prepare a 
June 1992 metrication status report to the Department of Commerce. In drafting these 
reports we used our knowledge gained from familiarity with NASA’s transition plan, 
participation at meetings, and contacts with the NMPG. Our efforts for OSMA have 
contributed to good relations with the Congress, the GAO, and the metric program office 
at NIST. 

Vitro supported and documented meetings of the NMPG that are scheduled on an as- 
needed basis. We also assisted OSMA with about one meeting per year for the Field 
Installation coordinators, and teleconferences as needed. 


ADVANCED TECHNOLOGIES 

OSMA manages applied technology programs for aerospace batteries and pyrotechnically 
assisted systems. These programs require preparation of program plans, handbooks, and 
other documents, development of databases, and general technical and administrative 
support for meetings and workshops. Vitro worked with the managers of applied 
technology programs to plan meetings and other support activities. Our staff helped 
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OSMA develop, review, and revise key documents. We also provided advice to the 
NASA program managers to ensure that NASA pursues relevant development activities. 

For the pyrotechnically assisted systems (PAS) program, Vitro assisted the OSMA 
program manager with the development of both a program plan and a program 
implementation plan. Vitro participated in PAS Steering Committee meetings and 
prepared minutes of these meetings. Vitro also participated in meetings of the 
aerospace battery program Steering Committee and undertook the task of publishing the 
proceedings of the 1987 and 1988 NASA Battery Systems Workshops. 


SOFTWARE ENGINEERING AND ASSURANCE 
Avionics System Software 

For each Shuttle flight, Vitro prepared a Shuttle Software Assessment in Support of the 
Flight Readiness Review (FRR). This effort evaluated software changes, discrepancies, 
and late flight software patches. The reports contained detailed descriptions for the 
Shuttle Avionics, Space Shuttle Main Engines (SSME), and Ground-Launched System 
Software in support of the Flight Readiness Review (FRR) and the Safety and Mission 
Assurance Certification of Flight Readiness. 

During the development of new Operational Increment (OI) flight software, Vitro 
reviewed software changes and discrepancies in all program phases, including initial 
requirements, development, embedded verification and validation, and final operation, 
and prepared technical assessments. For example, in November 1992, Vitro provided a 
status report and analysis of the major software Change Requests during baseline, 
development and mission preparation phases of the 01-22 flight software that was 
scheduled to fly with STS-57 in April 1993. Vitro maintained and updated OSMA’s 
flight software manifest relating the Shuttle flight schedule to significant software 
changes; this chart was delivered to Codes Q and M for monthly program reviews. 

Vitro also evaluated and analyzed software and hardware issues for the Shuttle avionics 
systems, performed independent assessments of problems, and recommended corrective 
action to OSMA. For instance, Vitro provided assessments of software development and 
independent verification and validation requirements during the PDR and CDR reviews 
for the new Day-Of-Launch I-Load Update Version II (DOLILU II) Processor. In 
addition, we reviewed and analyzed the development of Multifunction Display Electronic 
System (MEDS) software and hardware during the PDR and informed OSMA of issues 
and concerns, and reviewed and assessed the implementation of the Global Positioning 
System (GPS) project for the orbiter. 

Deliverables: 

"Shuttle Software Assessment," separate reports for each STS flight (49, 50, 46, 47, 52, 

53, 54, 56, and 55) manifested with the Operational Increment (01-21) flight software. 
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"Shuttle Software Assessment," separate reports for each STS flight (57, 51, 58, 61, and 
60) manifested with the Operational Increment (01-22) flight software. 

"Software Defect Analysis Assessment," VC-OSC-T-017(92), dated April 24, 1992. 

"Software Change Request Implementation Process Failure Assessment," VC-OSC-T- 
018(92), dated May 29, 1992. 

"STS-49 On-Orbit Flight Software Anomalies Report," VC-OSC-T-020(92), dated May 
29, 1992. 

"DOLILU II Preliminary Design Review (PDR) Status," VC-OSC-T-021(92), dated June 
19, 1992. 

"Autoland Detailed Test Objective (DTOJ Status," VC-OSC-T-026(92), dated August 31, 
1992. 

"Autoland Detailed Test (DTO) Status," VC-OSC-T-029(92), dated September 22, 1992. 

"STS-52 Late Flight Software K-Load Patch," VC-OSC-T-033(92), dated October 16, 
1992. 

"STS-52 Late Flight Software Patch Status Report," VC-OSC-T-033(92), dated October 
16, 1992. 

'Technical Assessment, Flight Software Operational Increment (01-22) Status Report," 
VC-OSC-T-036(92), dated November 23, 1992. 

"Assessment of AP-101S General Purpose Computer (GPC)," VC-OSC-T-070(92), dated 
January 2, 1993. 

"DOLILU n Critical Design Review (CDR) Status," VC-OSA-T-008(93), dated February 
22, 1993. 

"DOLILU n Delta Critical Design Review (CDR) Status," VC-OSA-T-028(93), dated 
August 2, 1993. 

"GPS Software Implementation Status Follow-up," VC-OSA-T-OOl (94), dated January 6, 
1994. 

"DOLILU II System IV&V Implementation," Report, dated March 19, 1992. 

"RMS Fault-Detection Error Annunciations," Report, dated May 11, 1992. 

"STS-49 Orbit Rendezvous Fault Message Annunciation," Report, dated May 14, 1992. 
"SASCB Discussion, Orbit Targeting Implementation," Report, dated July 13, 1992. 
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"PASS and BFS DRs (108538 and 108539) Possible Main Engine Bell Collision," Report, 
dated October 6, 1992. 

"KCR Tracking and New Implementation Procedure," Report, dated November 6, 1992. 

"SAIL Integration Avionics Verification for the 01-22 FSW," Report, dated April 19, 
1993. 

"STS-57 FSW Patch Threats," Report, dated March 11, 1993. 

"STS-71 Spacelab - MIR Mission Docking," Report, dated May 17, 1993. 

"Engineering Directorate FSW Process Review for 01-22 baselined (CR 90243C)," 
Report, dated June 8, 1993. 

"Implementation of Shuttle GPS Software," Report, dated June 8, 1993. 

"Assessment of a Change Request (CR) for Major Flight Software Enhancements to OI- 
23, 24, 25, and 26," Report to Codes Q and M dated July 19, 1993. 

"BFS NO-GO and Suspect Changes," Report, dated August 20, 1993. 

"DR 108676 BFS Unable to Transmit on Flight Critical Buses While Engaged," Report, 
dated October 21, 1993. 

"CR 90476 A - Wraparound Yaw Jet System W/Program Test Input (PTI) Effector," 
Report, dated November 16, 1993. 

"Presentation on Overall Review of the MEDS Project to OSMA," Presentation, dated 
November 18, 1993. 

"STS-58 BFS Payload TMBU Processing Error," Report, dated November 23, 1993. 
Software Assurance 

During the entire contract period, Vitro provided comprehensive support for the initial 
Software Engineering Program and the Software Assurance Program plans. An early 
effort assisted NASA in preparing, coordinating, and publishing NMI 2410.10, the basic 
NASA policy for software management, assurance, and engineering. Various Software 
Management and Assurance Program (SMAP) conferences, meetings, teleconferences, 
and videoconferences were supported. Software engineering expertise and experience 
was provided for developing, coordinating, editing, publishing, and disseminating software 
engineering and assurance documentation, including: 

• NASA Software Acquisition Life-Cycle 

• NASA-STD-2 100-91, NASA Software Documentation Standard 

• NASA-STD-2201-93, Software Assurance Standard 
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• SMAP-GB-A201, Software Assurance Guidebook 

• NASA-STD-2202-93, Software Formal Inspections Standard 

• NASA-GB-A302, Software Formal Inspections Guidebook 

• SMAP-GB-A301, Software Quality Assurance Audits Guidebook. 

Technical reports were prepared, as required, such as the Software Configuration 
Management Standard Study, October 1993. A recent emphasis was the identification 
and evaluation of voluntary industry standards, standards from other Federal agencies, 
and international standards regarding software on a continual basis. Where appropriate, 
use of applicable voluntary standards was recommended. 

ELECTRONIC PARTS PACKAGING 

Vitro coordinated, edited and distributed three quarterly reports covering activities of the 
RELTECH Committee comprising Air Force Rome Laboratory, Naval Surface Warfare 
Center, U.S. Army Research Laboratory and NASA. To accelerate issuing reports, a 
new format was developed for reporting activities and results. Vitro participated in 
various RELTECH meetings such as the Committee meeting in Denver (August 1993) 
and RELTECH/ICWG meetings in New Orleans (November 1993) and Fort Lauderdale 
(January 1994). Minutes for each Committee meeting were prepared, edited, and 
distributed. 

Vitro participated in RELTECH technical assessments/surveys of the following 
companies: 

• Texas Instruments (Memory Cubes and MCM Foundry), Dallas, TX 

• IBM, East Fishkill, NY 

• Martin Marietta, Orlando, FL 

• Integrated System Assemblies, Woburn, MA. 

At our suggestion, RELTECH completely revised its survey activities and developed 
guidelines specifying what areas and subjects are to be covered and the type of 
information required. 

Vitro attended and participated in the following technical conferences and meetings: 

• MCM-D & L, Ogonquit, ME, June 1993 

• Advanced Microelectronics Qualification/Reliability Workshop, August 1993 

• International Electronics Packaging (IPC) Conference, September 1993 

• Electronics Industry Quality Council, October 1993 

• International Society for Hybrid Microelectronics, November 1993 

• Government Microcircuits Applications Conference, November 1993. 

We gave the NASA presentation at the Workshop on MCM-L Reliability, held in 
conjunction with the 1993 IPC Conference. 
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ELECTRICAL, ELECTRONIC, AND ELECTROMECHANICAL (EEE) PARTS 


Support for the EEE Parts Program area included technical assistance and expertise in 
the preparation of procedures and policies, evaluation of Center and contractor 
programs, and technical support for technical meetings. (This report covers EEE parts 
activity after this technical area was added to this task; prior activity is covered in the 
Task Order 1000 final report.) Technical leadership and guidance was provided in the 
restructuring of the NASA Parts Steering Committee (NPSC), EEE Parts Research and 
Technology Projects, and the EEE Parts Radiation Program. This effort was 
accomplished through the development of new NPSC Committee charter and Briefing 
Book format, the development of the Integrated Radiation Hardness Design Assurance 
(IRHDA) working group charter and strategic plans. Technical direction was provided 
in redirecting the large JPL and GSFC RTOP programs to be more responsive to NASA 
Center and NASA program needs. Vitro also provided technical direction was provided 
in redirecting and eliminating program waste from the EEE parts radiation programs 
and GSFC and JPL, and restructuring the programs into a multi-center IRHDA working 
group to share program costs between Headquarters, Centers, and other federal space 
agencies. The restructuring of the NPSC and IRHDA program areas accomplished the 
integration of similar program elements at each Center into a coordinated strategic 
effort, emphasized a customer orientation of the EEE Parts Program elements, 
consensus of decision making with participation from all NASA center EEE parts and 
Radiation community experts, and the direct application or transition of center field 
problems into the EEE Parts and Radiation program elements. 

The multi-million dollar "EEE Parts and Radiation Effects" RTOP program at JPL and 
GSFC was brought under strategic planning control, and included each of the NASA 
participating centers in the NPSC. The strategic planning activity was evaluated in terms 
of the changing objectives of the EEE Parts program and new implementation of the 
IRHDA program, establishing objectives and deliverables designed to meet the 
requirements of all NASA Centers (not just JPL and GSFC), and evaluating the 
individual program elements of the EEE Parts and IRHDA programs to determine their 
successfully meeting the stated NASA-wide center and program needs. This resulted in a 
broadening of Code Q support to other Centers to improve the return-on-investment of 
the EEE parts and IRHDA RTOP programs, which had been an ongoing area of 
disagreement and dissatisfaction from the viewpoint of the SRM&QA Directors at the 
NASA Field Installations and Program Directors of Space Station Freedom (SSF) and 
Space Transport System (STS). 

Vitro provided technical assessments and oversight to the ongoing systems development 
effort of the Electronic Parts Information Management System (EPIMS). Planning and 
funding decisions were coordinated between Vitro and NASA Code Q management 
elements to review and coordinate complementary funding and management initiatives 
between the EPIMS and its mechanical parts equivalent, Mechanical Parts Information 
Management System (MePIMS). Coordination requirements and planning meetings 
were held with representatives of all NASA Field Installations to address system 
planning deficiencies not addressed by GSFC Center management of EPIMS and 
MePIMS. 
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Vitro also performed assessments of the impact of power transistor (2N3421 - Unitrode) 
and tantalum capacitor (Sprague) failure scenarios for Mars Observer failure 
investigation team at NASA HQ and Naval Research Laboratory (NRL) (September- 
October 1993). 

Deliverables: 

"Electrical, Electronic, and Electromechanical (EEE) Parts Management and Control 
Requirements for NASA Space Flight Programs," NHB 5300.4(1F), July 1989, Editor and 
Contributor. 

"Implementation of NASA Standard Electrical, Electronic, and Electromechanical (EEE) 
Parts Program," NMI 5320.6A, revised NMI 5320.6B, October 1991, Final Draft. 

"Basic Policy for NASA Space Flight Program Electrical, Electronic, and 
Electromechanical (EEE) Parts," NMI 5320.5A, revised NMI 5320.5B, May 1991, Final 
Draft. 

"EEE Parts and Reliability: Data Sharing Survey," Briefing and Stand-alone Data Survey 
Compilation Diskette, October 1993, for distribution to ALAA/NSIA Coordination 
Meeting and US Space Parts Strategic Steering Committee. 

"NASA EEE Parts Program - Integrated Radiation Hardness Assurance Program 
(IRHDA) Independent Review Briefing Book," August 1993. Editor and Contributor. 

"NASA EEE Parts Program Briefing Book - Review and Prioritization of EEE Parts 
Program Tasks," NPSC Meeting, Lewis Research Center (LeRC), July 27-28, 1993. 

Editor and Contributor. 

"Radiation Hardness Design Assurance For Space Electronics Systems In The ‘Better, 
Faster, Cheaper’ Era," May 1993 - August 1993. Editor and Contributor. 

"NASA Strategic Committee on Space Parts Radiation Hardness Assurance," February 
1993. 

"Space Electronic Parts Infrastructure Assurance (SEPIA): Development Strategy," 
September 1992. 

"Parts Manufacturer Survey Evaluation Guide Book," Final Draft, February 1994. 

"RADATA DATABANK: A Full Text Retrieval Infobase," Folio VIEWS data retrieval 
version of JPL’s RADATA Databank, prepared by Vitro and Labat-Anderson, October 
1991. 

"GIDEP DATA Base: A Full Text Retrieval Infobase," Folio VIEWS data retrieval 
version of GIDEP Alert database, prepared by Vitro and Labat-Anderson, October 1991. 
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Grade 2 Parts for the EOS Program," December 1992. 


III. CONCLUSIONS AND RECOMMENDATIONS 

Over the life of Task Order 4000, Vitro has been responsive to OSMA’s changing 
requirements. We have reorganized our staff to make our organization a reflection of 
OSMA’s. While the scope of our effort has been altered to reflect OSMA’s budget 
reality, we have made significant contributions to each technical area. These 
accomplishments, described above, have enabled Code QW to expand its budget and 
staff and to better serve other NASA programs. Based on our experience working with 
Code QW, plus our familiarity with the other NASA programs, we make the following 
recommendations to Code QW: 

• Engineering Standards'. The use of appropriate standards is the foundation of 
NASA’s assurance capability. NASA and Code QW need to increase use of 
voluntary and DoD standards, reduce reliance on institutional standards and, where 
necessary, develop Agencywide standards. Code QW should complete the 
development of and then implement an improved process for developing standards 
and establish a management system for NASA standards. 

• Metric Transition : Code QW needs to update the NASA Metric Transition Plan to 
reflect the current program responsibilities, budgets, and schedules. Joint efforts 
with other government agencies should be expanded to make better use of limited 
resources and yet make the fastest reasonable progress on the adoption of the 
metric system. 

• Advanced Technologies : Code QW currently supports selected advanced 
technologies. Code QW should consider developing a well-defined procedure for 
identifying new areas that deserve support. 

• Systems Engineering: Code QW participates in several system engineering 
initiatives, including the Systems Engineering Process Improvement Team (SEPIT), 
the System Engineering Working Group, the Program Excellence Team, and the 
National Initiative for Product Data Exchange. Code QW needs to maintain basic 
program management information (e.g., goals and objectives, deliverables, schedule, 
and budget) on the status of cooperative programs, specifically in a form 
appropriate for management briefings. In addition, Code QW must identify 
national systems engineering issues and additional opportunities for cooperative 
programs. 

• Software Engineering and Assurance : Code QW’s goal is to assist program planning 
to the critical evolving areas of NASA software safety, engineering, assurance, and 
IV&V technology development. The effort covers NASA’s program life cycle, from 
requirements definition, through design, test, and independent verification and 
validation (IV&V), to operations and maintenance. In the future, Code QW should 
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bolster its technical expertise for directing the IV&V Facility, technical assessments, 
and preparation of software standards, guidelines, and metrics. 

• Electronic Parts Packaging and EEE Parts: Code QW should consider integration 
of these activities into a broad parts, materials, and processes effort. 
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5000 SERIES TASK ORDER 

QUALITY AND PRODUCTIVITY AWARENESS PROGRAM/SRM&QA CAREER 
DEVELOPMENT AND TRAINING 


I. BACKGROUND FOR THE OVERALL TASK 

After contract award, it was readily apparent to OSMA that a vehicle was needed to 
obtain support for the Quality and Productivity Improvement Programs Office (Code 
QB). Hence, a Quality and Productivity Awareness Program/SRM&QA Career 
Development and Training task order was established to provide this support. Code QB 
was officially established by the NASA Administrator to operate under the Associate 
Administrator for Safety, Reliability, Maintainability, and Quality Assurance (Code Q). 
The Office plays a significant role in advancing NASA’s image as a leader in national 
quality andproductivity by promoting quality and productivity improvement programs 
within NASA and its contractor community and sharing these improvement initiatives 
with other government agencies, educational institutions, the business and industry 
community, and professional associations and societies. The task order required Vitro 
Corporation to assist Code QB in establishing NASA as a center of excellence through 
firm support of Code QB’s quality and productivity initiatives; in particular, the 
establishment of efficient and effective training and development programs for NASA’s 
SRM&QA personnel. 


II. SUMMARY OF TASK ACTIVITY 

Vitro provided technical and administrative support for the SRM&QA Career 
Development and Training (CD&T) Program and for the Quality and Productivity 
Awareness (Q&PA) Program. 

SRM&QA Career Development and Training (CD&T) Program 

The SRM&QA Career Development Program (CDP) Plan involved implementation of 
an agency SRM&QA CD&T Program. The primary objective of the program was to 
provide a structured framework for the training and development of a sufficient number 
of SRM&QA engineers and technical specialists to meet NASA’s immediate and 
projected SRM&QA workforce requirements. A secondary objective was to provide, to 
the maximum extent possible, career opportunities that generally satisfy the personal 
aspirations of the agency’s SRM&QA employees. Therefore, the NASA SRM&QA CDP 
Plan was designed to establish NASA as a center of excellence in the training and 
development of SRM&QA personnel, meeting both individual and Agency needs. Vitro 
planned, coordinated, and implemented all activities associated with the SRM&QA 
CD&T Program. Vitro supported and coordinated the Career Development Working 
Group (CDWG) activities. Vitro also assisted, as required, the NASA Centers in the 
formulation of center career development and training plans and assisted Code QB in 
monitoring center career development activities. 
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The NASA SRM&QA CDP Plan required each Center to have its own career 
development program or system in place and in writing. This latitude was designed to 
allow the Centers to tailor their career development program activities to fit center- 
unique needs. Also, each Center’s SRM&QA Director was required to report the status 
of that Center’s career development activities to the Associate Administrator for 
SRM&QA each year. The NASA SRM&QA CDP Plan also specified the format for the 
report. 

Vitro also assisted Code QB in identifying and analyzing agency training and 
development requirements and sources of training and development activities to fulfill 
those requirements. Vitro developed a mechanism for coordinating the planning and 
development of agency SRM&QA training so as to more effectively identify unfunded 
training requirements and existing training course development initiatives that would 
fulfill those requirements. Vitro collected^ analyzed information, and made 
recommendations regarding SRM&QA CD&T Program improvements, as required by 
Code QB. The primary sources of information were other government agencies, 
educational institutions, and industry, as well as the broad base of experience within 
NASA. 

Vitro conducted a survey of government and industry SRM&QA Training and 
Development (T&D) Programs. The Survey of SRM&QA Work Force Training and 
Development Programs in Selected Industrial Establishment and Government Organizations 
report was designed to document successful long-term T&D programs used in aerospace 
and aerospace-related industries for all personnel in general and SRM&QA personnel in 
particular. Vitro contacted a wide variety of industrial and government organizations 
that have demonstrated successes in SRM&QA and related technical fields to describe 
their SRM&QA T&D programs and activities. The survey report provides important 
information to be used to refine the NASA SRM&QA Career Development and 
Training Programs, as well as in aiding other organizations in designing their own 
SRM&QA T&QA programs. These long-range T&D strategies play a major role in 
ensuring a sufficient supply of professional engineers and discipline specialists for 
SRM&QA. 

Vitro also established and maintained an automated SRM&QA Training Courses 
Compendium (TCC) Data Base. The TCC Data Base is a compendium of courses, 
degree and intern programs, documents, and organizations associated with safety, 
reliability, maintainability, and quality assurance, and such related topics as risk 
analysis/management, integrated logistics support, and configuration management. This 
data base was designed for use in association with the NASA SRM&QA CDP and other 
SRM&QA T&D efforts by assisting supervisors and training personnel in determining 
sources of SRM&QA T&D activities and materials. Following data base program 
development and validation, data entry was accomplished. Vitro coordinated and 
installed the TCC Data Base at all NASA Code Divisions and shipped the TCC Data 
Base to all NASA Centers complete with the Installation and User’s Guide. 

Vitro also established a SRM&QA CD&T Library to provide a resource center of 
materials to support NASA’s SRM&QA CD&T activities. These materials include 
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general books on career development, career counseling, training, and similar topics; 
college catalogs outlining numerous SRM&QA course and degree programs; catalogs 
and pamphlets regarding NASA professional development opportunities at Headquarters 
and the centers; brochures concerning commercially available SRM&QA training; and 
related items. The library was located at Vitro’s Maryland Avenue office and was open 
for use by NASA personnel on a walk-in basis. Library materials could also be checked 
out to NASA personnel at field center locations. 

Quality and Productivity Awareness (Q&PA) Program 

The Quality and Productivity Awareness (Q&PA) Program included numerous quality 
and productivity initiatives for which Code QB was responsible. Vitro assisted Code QB 
in promoting these initiatives. Promotional activities were limited to the preparation of 
articles, presentation material, posters, brochures, and pamphlets. Vitro arranged for the 
development and preparation of a Quality and Productivity presentation package 
complete with graphics/viewgraphs for use by the Code QB Director. Vitro also 
arranged for the development of a Quality Awareness Training Program for NASA 
Headquarters Code Q Supervisors and the conduct of such training. 

In addition. Vitro provided technical support for preparation of documents and 
publications such as the Quality and Productivity Awareness Newsletter. This support 
entailed creative writing/editing of technical and quality awareness material as required 
by the Q&PA Program. Vitro also established and maintained a reference library of 
books, pamphlets, documents, and papers in support of the Q&PA Program. 


III. CONCLUSIONS AND RECOMMENDATIONS 

The Quality and Productivity Awareness Program/SRM&QA Career Development and 
Training task orders spanned the period of February 22, 1988 through August 15, 1989, 
at which time it was transitioned to another contractor, Information Dynamics, Inc. (IDI) 
under a SBA 8(a) set-aside contract. Milestone charts (see attachment) for this period 
detail the activities and status of all deliverables. All government-furnished material 
held by Vitro Corporation in support of the Quality and Productivity Awareness 
Program/SRM&QA Career Development and Training task order was transferred to 
IDI, with Mr. Arthur Palmer of the NASA Code QB Office acknowledging receipt on 
August 15, 1989. 

Due to the decentralized nature of the SRM&QA Career Development and Training 
(CD&T) Program, the success or failure of the overall Agencywide NASA SRM&QA 
CDP will depend heavily on training and development efforts at the Centers. This places 
a heavy burden on the Centers, since they must develop, implement, and support a viable 
CDP, and on Code Q, since it must measure success or failure largely through its 
evaluation of the Center programs. It was Vitro’s opinion that, at the time of task 
transition to IDI, two center CDP plans were excellent (ARC, LeRC); two were good 
(KSC, SSC); two were inadequate (GSFC, MSFC); and two were not yet developed 
(JSC, JPL). One Center (LaRC) insisted that it would not participate formally. All 
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Centers must develop program plans as viable as those at ARC, LeRC, KSC, and SSC 
for the overall Agencywide program to succeed. 
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5000 SERIES TASK ORDER 

SOFTWARE INDEPENDENT VERIFICATION AND VALIDATION SUPPORT 


I. BACKGROUND FOR THE OVERALL TASK 

In 1992, NASA began construction of a Software Independent Verification and 
Validation (IV&V) Facility in Fairmont, West Virginia. The Facility serves as a NASA 
resource and center of excellence for software IV&V research, application, and training 
that enhance project support to the NASA mission. This series of task orders was 
created to provide the IV&V Facility (Code QV) with software IV&V expertise to assist 
in the development, integration, and operation of the Software IV&V Facility. Vitro 
provided software IV&V support from November 8, 1993 through February 10, 1994. 


II. SUMMARY OF TASK ACTIVITY 

The Vitro software IV&V staff personnel assisted in planning and development of the 
first NASA IV&V workshop, which was held December 13-16, 1993 and attended by 
over 150 software IV&V experts. This subtask entailed defining the scope and thrust of 
the workshop, identifying and obtaining commitments from candidate speakers and 
workshop leaders, preparing announcements, and other assistance as required. Vitro 
personnel also was asked to co-chair the Criticality session. 

Vitro also assisted in preparing a Software IV&V Strategic Plan to integrate the overall 
requirements and proposed products associated with the Software IV&V Facility. 

Vitro was asked to assist in developing an interim Statement of Work for the 
International Space Station Alpha IV&V effort to be performed by the Software IV&V 
Facility. This subtask entailed planning activities to be performed between February 
1994 and September 1994. 


III. CONCLUSIONS AND RECOMMENDATIONS 

Deliverables were accurate and timely, and fully satisfied NASA requirements. The 
requirements of task order were well defined; therefore, no recommendations are 
provided. 
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6000 SERIES TASK ORDER 
ADMINISTRATION 


I. BACKGROUND FOR THE OVERALL TASK 

After contract award, it was readily apparent that a vehicle was needed to obtain the 
administrative and financial reporting requirements associated with the contract. Hence, 
a separate task order for Administration was established. The Administration task order 
encompassed the full spectrum of administrative and financial support needed to provide 
quick response to NASA Code Q requirements. The Administration task order required 
Vitro Corporation to employ personnel with expertise in task order management, 
budgeting and cost performance reporting, subcontract/consultant administration, 
resources management, logistics, security program administration, govermnent property 
administration, facilities management, information management system, and conflict of 
interest avoidance. The Administration task orders spanned the period of February 22, 
1988 through February 10, 1994. Staff support task orders (formerly 8000 series) became 
a part of this series in August 1992. 


II. SUMMARY OF TASK ACTIVITY 

Vitro Corporation Administration staff personnel provided financial reports as required 
under NASA Headquarters contract NASW-4311 and associated task orders using the 
Contractor Financial Management Report (NASA Forms 533M and 533Q) and a 
specially generated and formatted biweekly report (see attachment). The Contracting 
Officer (CO) and the Contracting Officer’s Technical Representative (COTR) approved 
the reporting formats in content and design, and also established the delivery schedule. 
The biweekly reports provided a cost breakdown by major elements, and included 
manpower and budget information required to successfully manage the contract. On 
their own initiative, Vitro Administration staff personnel also designed, developed, and 
produced budget and cost graphs (see attachment) on a biweekly basis. The graphs were 
extremely useful in managing and controlling contract costs as trends were easily 
identified. The COTR readily recognized the value of the graphs and made them a task 
order deliverable. The standard NASA monthly and quarterly financial reports, 
specialized biweekly budget and cost reports, and biweekly budget and cost graphs 
enabled NASA to properly monitor contractor cost performance at the task order and 
job summary level (hours and cost). The financial reports also furnished the Vitro 
Program Manager and his staff with the data needed for planning, budgeting, and cost 
control. The quality and timeliness of these reports were superior. 

Vitro established a system for identifying and tracking NASA Safety, Reliability, 
Maintainability, and Quality Assurance (SRM&QA) contractual conflicts of interest. A 
procedure was established to identify all potential conflicts of interests in the pursuit of 
new business and to obtain CO approval. No conflict of interests were experienced 
throughout the performance of the contract. 
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In the area of subcontract/consultant administration, Vitro was highly effective in 
obtaining specialized short-term support in a timely and effective manner through 
subcontract/consultant agreements in areas where we did not have the in-house 
expertise. 

On a monthly basis. Vitro Administration personnel updated, reviewed, and submitted 
the Government Property Report to the Corporate Property Office for subsequent 
submittal to the Government. All Government equipment was returned to NASA prior 
to the contract completion date. 

After the merger of SRM&QA support (see 8000 series task order) into this task order 
in 1993, Vitro continued to provide technical writing and editing support for 
management reports, presentations, and other special projects for the Code Q 
management and engineering staff. Vitro also provided technical editing, word 
processing, and publications preparation support for assigned documents generated 
for/by Code Q management and engineering staff as described in other tasks, including 
NASA Management Issuances, NASA Handbooks, position papers, and technical reports. 
Vitro also assisted management and engineering staff in the preparation of Code Q 
presentation materials (viewgraphs, slides, brochures, insets, etc.) for management 
reviews and technical presentations. Critical deliveries included NHB 1700.1, "Safety 
Policy and Requirements Document," NSS 1740.12, "NASA Safety Standard for 
Explosives, Propellants, and Pyrotechnics," and quarterly reports summarizing the 
accomplishments of the RELTECH Steering Committee and Teams. 

Vitro also continued its administrative support to the Aerospace Safety Advisory Panel 
(ASAP). The main thrust of this support focused on assisting the Panel in generating the 
ASAP Report. In 1993, Vitro employees were recognized for excellent publication 
support to the Space Shuttle Main Engine (SSME) Assessment Team, an ad hoc task 
forced created by ASAP to conduct a thorough assessment of the risks that the SSME 
poses to the safe operation of the Space Shuttle. 

In the latter phase of the contract, Vitro provided support for the maintenance and 
upgrade of administrative data systems (e.g., budget systems, action tracking systems, 
etc.) to meet Code Q management and engineering requirements. 


III. CONCLUSIONS AND RECOMMENDATIONS 

The deliverables were accurate and timely, and fully satisfied NASA requirements to 
monitor the SRM&QA contract to the task level. Additional support in tangible areas 
other than scheduled requirements was provided in an expedient and professional 
manner. The majority of the requirements of the Administration task order were well- 
defined as they are standardized throughout NASA. This is especially the case in the 
area of financial reporting. Therefore, no recommendations are provided. 
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7000 SERIES TASK ORDER 
PROGRAM MANAGEMENT 


I. BACKGROUND FOR THE OVERALL TASK 

Shortly after contract award, the NASA Code Q management requested that a separate 
task order for Program Management be established whereby NASA Code Q would have 
more direct visibility into the overall Vitro management activities at a program level. 
This separate task order also gave NASA Code Q a more direct instrument under which 
to evaluate the Vitro Program Manager’s performance and effectiveness. 


II. SUMMARY OF TASK ACTIVITY 

The Program Management task was essentially one of managing the entire Vitro support 
effort, paying particular attention to providing responsive and quality technical support 
within budget. This was fully accomplished during the contract period. 

Vitro provided all the technical, engineering, and management skills/disciplines 
necessary to assist NASA Code Q in its overall mission being responsive to task direction 
from Code Q. Vitro performed these functions with a skill mix that continuously evolved 
to fulfill changing Code Q requirements as Code Q itself adapted to a changing NASA 
environment. Vitro adapted its organizational structure and staffing throughout the life 
of the contract to reflect the changes required by Code Q. Vitro assigned personnel with 
indepth experience attuned to the needs of Code Q. When needs changed, Vitro rapidly 
responded by utilizing Vitro corporate resources and subcontracted with highly talented 
specialists or other contractors. This enabled Vitro to quickly adjust staffing levels and 
skills to meet short-term workload variations and requirements. 

Responsiveness to Code Q was crucial and Vitro was most effectively located within 
close walking distance to NASA Headquarters. The ability to personally interface with 
Code Q staff on a daily basis without losing productive time in travel was invaluable and 
advantageous to Code Q. 

Vitro gained an understanding of the "protocol" involved (e.g., that OSMA must approve 
and in some cases make the initial contact) when interfacing on a technical level with 
elements at a Field Installation or with associated program-related contractor personnel. 
Vitro established numerous important SRM&QA points of contact at the nine NASA 
Field Installations and with supporting SRM&QA and program contractors and used 
these contacts on a daily basis to support Code Q. 

Vitro developed significant integration processes and actively participated in Space 
Shuttle SRM&QA continuous process improvement-related activities at the NASA 
manned spaceflight Field Installations. Space Shuttle enhancement reviews suggested by 
Vitro were implemented to integrate long-term SRM&QA Field Installation process 
improvement efforts. Vitro produced the Space Shuttle Safety and Mission Assurance 
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(S&MA) Operating Plan and a process improvement matrix for Space Shuttle S&MA to 
assist in controlling process improvement activities. Vitro helped develop a Code Q 
Process Improvement Plan to help integrate Code Q process improvement efforts and to 
advocate SRM&QA improvements to the Space Shuttle program. 

Vitro assisted Code Q in assessing and communicating complex Space Station Electrical 
Power System (EPS) stability issues to Space Station program managers. Vitro 
performed critical design evaluations of the Day-Of-Launch I-Loads Uplink (DOLILU) 
processor systems software development, providing OSMA with a strong rationale for 
advocating an Independent Verification and Validation (IV&V) activity leading to the 
Space Shuttle Project Office funding a full IV&V effort. Vitro assisted Code Q in 
identifying critical dynamical issues related to the rendezvous and docking aspects of the 
forthcoming U.S./Russian Spacelab/Mir Mission (SLM). Vitro also introduced the idea 
for and assisted Code Q in implementing jin integrating function, the Potential 
Issue/Problem Identification Committee (PIPIC), for independent assessments that 
require coordination and support across a number of different functional areas. Vitro 
developed procedures/software for document management that are now in use within 
Code Q. 

Vitro personnel have received over 200 awards, commendations, and recognitions from 
Code Q and other NASA and NASA-related industry organizations. Some of these 
awards are for engineering support to technical interagency panels such as the Aerospace 
Safety Advisory Panel (ASAP), the Interagency Nuclear Safety Review Panel (INSRP), 
and the Aerospace Industries Association (AIA)/National Security Industrial Association 
(NSIA)/NASA Liaison Panel. Further awards resulted from our support to the Jet 
Propulsion Laboratory (JPL) Radiation Effects and Testing Group, where Vitro 
engineers helped JPL in their increased focus on electronic, electrical, and 
electromechanical (EEE) parts. The Vitro Safety and Engineering Assurance Group 
twice received the NASA Headquarters Special Service Group Honor Award. Vitro 
personnel also received personal recognition for technical assessments and support to 
Code Q, which included commendations for risk management/assessment and 
probabilistic failure analysis recognized at NASA Headquarters and Marshall Space 
Flight Field Installation, technical support to the LOX/LH2 Explosion Hazards program, 
technical assessments on space flight data recorders and qualification of hardware by 
similarity, and development of software policies and standards. Vitro conducted a 
prelaunch assessment of cracks in the hinge of the External Tank doors on STS-37R. 

This assessment provided Code Q with technical rationale to delay the launch until 
problems were corrected. The Vitro engineer who performed the assessment was later 
honored with the Manned Flight Awareness Award, one of five such prestigious Manned 
Flight Awareness Awards received by Vitro personnel during the tenure of the contract. 


III. CONCLUSIONS AND RECOMMENDATIONS 

The Vitro contract effort has been managed extremely well. Vitro overcame a start-up 
problem to provide outstanding support to NASA Code Q, providing them with special 
and unique SRM&QA expertise not resident within NASA Headquarters. During the 
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entire period of the contract, Vitro’s effort was always under budget. In addition, Vitro 
has had eight "excellent," two "very good", and one "good" award fee ratings over the 
duration of the contract,. The "good" rating was for the very first 6-month period when 
Vitro was in the startup phase of the 6-year effort. It should be especially noted that 
Vitro’s "excellent" performance was maintained for a majority of a significant 
"downsizing" period, in which the Vitro staffing was essentially reduced from a high of 
approximately 102 persons to a final staffing of approximately 48 persons, a 53% cut in 
personnel. During these trying times, Vitro personnel maintained extraordinary 
professionalism, continuing to provide outstanding support to NASA Code Q. 

A significant factor in Vitro’s providing outstanding support to Code Q has been the 
sense of "teamwork" instilled amongst NASA and Vitro personnel. We are firmly 
convinced that the "team" concept begun under Mr. George Rodney and further 
enhanced/encouraged under Col. Frederick Gregory is the reason for not only Vitro’s 
outstanding performance, but also the successes attained by NASA Code Q. It is only 
when the people working on a program /project really feel that they are an essential part 
of it, do they perform at their best. Such a relationship must continue. 

Vitro encourages and recommends that NASA Code Q continue and expand upon the 
"team" concept already established between Code G and its support contractor. NASA 
should not establish a relationship that keeps its support contractor at "arms length"; 
"teamwork" leads to better communication, which in turn leads to increased performance, 
which ultimately benefits NASA and the contractor as well. 


74 



8000 SERIES TASK ORDER 
SRM&QA STAFF SUPPORT 


I. BACKGROUND FOR THE OVERALL TASK 

Shortly after contract award, OSMA (Code Q) management determined that a separate 
task order for SRM&QA Staff Support was needed. As a result, Vitro formed a 
Technical Support group was formed to provide administrative support directly to the 
Associate Administrator, Deputy Associate Administrator, and their immediate staff. 
Shortly thereafter, this support was matrixed across all task orders. In 1992, at the 
request of the COTR, all technical support was consolidated under Institutional and 
Resources Management (Code QM). The SRM&QA Staff Support task orders spanned 
the period of August 11, 1988 through August 10, 1992, at which time they were 
transitioned to the Administrative Support Task. 


II. SUMMARY OF TASK ACTIVITY 

During the contract period, Vitro provided office automation and local area network 
systems, technical editing, word processing, and publications preparation support for a 
wide range of assigned documents generated for and by the Associate Administrator, 
Deputy Associate Administrator, and Code Q division managers as described in other 
tasks. This effort included NASA Management Instructions and Handbooks, Safety 
Standards, Guidebooks, position papers, management plans, technical assessments, 
technical specifications for ADP/T products and services, and technical reports. Vitro 
also assisted management and engineering staff in the preparation of Code Q 
presentation materials (viewgraphs, slides, brochures, insets, etc.) for numerous high-level 
management reviews and technical presentations, and provided support for special 
studies. This work was frequently accomplished under very tight deadlines and with 
minimal direction from Code Q. 

Vitro supported and assisted in the development of several major Code Q speeches; 
namely, an "NDE for the Nineties" speech, Chapter 9 of the AJAA Handbook on 
Astronautics, the "Road Map To Excellence" speech for the NSIA Conference, the 'TQM 
in Perspective" and 'TQM — The Move From Product to Process Control" speeches for 
the First Total Quality Management Symposium, speakers’ notes and vugraphs for the 
NASA Management Training course, a speech for Mr. George Rodney for the Reliability 
and Maintainability Symposium; development of an independent study in response to 
Congress Roe’s letter concerning control of SRM&QA program funds, development of 
position papers on five Augustine Report recommendations, development of a technical 
paper/speech on Space Product Assurance in the 1990s for delivery at ESTEC; the 
"Testing in the Twenty First Century" speech for the 12th Aerospace Testing Seminar; 
the "Assuring Safety and Mission Success for Space Station Freedom" paper and speech 
for the International Symposium on Reliability and Maintainability; and the Critical 
Safety Assurance Factors for Manned Spacecraft" paper for the 41st Congress of the 
International Astronautical Federation (IAF). 
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Vitro also provided technical writing and editing support for the Hubble Space 
Telescope Board of Investigation and the Human Error Incidents review at KSC; and 
development of designs, color renderings, line-drawings, and 3-D concept models for the 
George M. Low, NASA Quality and Excellence Award, Trophy. 

Vitro provided administrative support to the Aerospace Safety Advisory Panel (ASAP). 
This effort consisted of providing editorial, graphics, original cover art as well as 
computer graphics, and publications engineering support to the Panel to assist them in 
developing the ASAP Annual Report and supporting the annual briefing to the 
Administrator. Vitro’s graphics staff developed a unique report cover depicting the 
particular theme of the year’s work. 

Vitro personnel provided support for the OSMA File/Records Conversion task. We 
were able to use our comprehensive knowledge of file management systems to assist in 
the development of Division file management plans. This laid the groundwork for Code 
Q to continue this task in-house. 

Vitro also provided support for the maintenance and upgrade of administrative data 
systems (e.g., budget systems, action tracking systems, etc.) to meet Code Q management 
and engineering requirements. 

The Vitro technical support staff was recognized for outstanding performance of critical 
writing and editing assignments. Included were the prestigious Manned Flight Award 
(MFA) and letters of commendations for outstanding and professional support to the 
Office of the Associate Administrator for Safety and Mission Quality (now Mission 
Assurance), Interagency Nuclear Safety Review Panel, and ASAP. One employee was 
cited for excellent writing support to NASA quality and productivity improvement 
programs. 


III. CONCLUSIONS AND RECOMMENDATIONS 

Deliverables for this task order were accurate, timely, and fully responsive to Code Q’s 
requirements. A significant factor in the success of Vitro’s Technical Support effort for 
Code Q has been the sense of "teamwork 1 ' between NASA and Vitro personnel. Vitro 
recommends that Code Q continue to expand and improve this "team" concept. 
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9000 SERIES TASK ORDER 

SPACE FLIGHT SAFETY AND MISSION ASSURANCE 


I. BACKGROUND FOR THE OVERALL TASK 

This task series was originally formed to provide the NASA Programs Assurance 
Division (Code QP) with engineering and technical support for the Space Shuttle 
Program, Space Station Freedom (Level I), Spacelab, Expendable Launch 
Vehicles/Upper Stages (ELV/US), Payloads, and for Aeronautics and Space Exploration 
activities. The primary purpose of this support has been to enhance Code QP’s ability to 
maintain a current knowledge and understanding of program status to support reviews of 
major problems, issues and concerns, and flight readiness. Code QP also required 
technical support for related program efforts such as the modification of program Safety, 
Reliability, Maintainability, and Quality Assurance (SRM&QA) documents. 

In December 1992, a Code Q reorganization resulted in the Payload Programs being 
transferred to the Quality Management (Payloads) Division (Code QT). In addition, the 
Space Shuttle safety group within the Safety Division (Code QS) was transferred to the 
Space Flight Safety and Mission Assurance Division (Code QP). The Division assumed 
the responsibility for the Mission Safety Evaluation (MSE) reports, and the safety tasks 
transferred within Vitro to this task order. 

In January 1994, the Expendable Launch Vehicle/Upper Stage (ELV/US) programs, the 
Aeronautics (Code R) programs, and the Advanced Concepts and Technologies (Code 
C) programs were transferred to the Payloads and Aeronautics Division (Code QT). 
Again, Vitro resources have been adjusted to support the new realignment of tasks for 
the NASA Code Q Divisions. 


II. SUMMARY OF TASK ACTIVITY 

Vitro’s approach has been to assign engineers, with directly related experience, to 
participate in reviews such as management, planning, engineering assessment, status, 
processing, change control, design, and flight readiness, with the objective of ensuring 
SRM&QA concerns were identified and appropriately addressed. Vitro also provided 
technical support for the modification of program documents, and sent qualified 
individuals to support all Level I/II Program Requirements Control Board (PRCB) 
telecons and related program technical meetings. Typical products were quick response 
technical assessments of SRM&QA issues and concerns, bi-weekly reports, monthly 
reports, trip reports, and special topic technical reports on significant issues, such as 
process-related problems. Vitro also modified numerous documents for Code QP, and 
recommended changes to program documents to enhance the SRM&QA requirements 
effectiveness. 
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Space Shuttle Program 

The Vitro engineering team has very been successful in providing technical assessments 
and reports of SRM&QA issues and concerns for the many Space Shuttle missions flown 
throughout the contract period. The NASA Associate Administrator for Safety and 
Mission Assurance (Code Q) commended the entire Vitro engineering support team for 
Code QP as follows: " The proof of your competence and one of the products that 
reflects your ability to succinctly summarize the key issues for top management, is the 
Biweekly Report." 

Space Shuttle documents, such as the Contingency Action Plan and the Launch and 
Mission Support Plan, have been significantly revised to reflect the changes in Code Q 
management and processes. Vitro engineers developed a Space Shuttle Program 
Enhancement Review Process plan and Operating plan to define a review process that 
would assist Code QP In advocating SRM&QA changes for the long-term enhancement 
of the Space Shuttle Program. 

One of the key Vitro Space Shuttle engineers developing these plans was honored as a 
Manned Flight Awareness Honoree for his outstanding technical contributions and 
judgment and the development of an Enhancement Review Process Plan in support of 
the Space Shuttle Program, thus contributing to each mission’s success. 

Space Shuttle Mission Safety Evaluation (MSE) Report 

Since the MSE and safety efforts were transferred to this task, the MSE report process 
has been evaluated using Continuous Process Improvement techniques to enhance the 
product. The results of two customer surveys and the process review led to a 
restructuring of the Vitro process to improve the quality and technical accuracy of the 
MSE report. Significant format changes have been made to facilitate Vitro engineers in 
updating and changing information, while reducing production costs. The Vitro engineer 
who managed this effort was recently honored by NASA as a Manned Flight Awareness 
Honoree, in recognition of his efforts to provide the SRM&QA and Space Shuttle 
Program communities with a single source of risk assessment information, as a 
contribution to each mission’s success. 

Space Station Freedom (Level I) 

The Space Station Freedom was restructured and redesigned in August 1993 as the new 
International Space Station Alpha. Much of the work completed during the many 
Preliminary (PDRs) and Critical Design Reviews (CDRs) held prior to that time is being 
re-evaluated in light of the new design and cost constraints. Some of the significant 
efforts Vitro engineers performed on CDRs for the Canadian Space Agency (Remote 
Manipulator System), and the Italian Space Agency (Mini Pressurized Logistics Module) 
will endure, since the International Space Station Alpha is not expected to impact these 
International Partner elements. The RIDs were written against the CDR documents to 
enhance the SRM&QA requirements effectiveness. Since the new contract does not 
require Space Station support, this work has been turned over to NASA. 


78 



Another significant Space Station effort involved the stability requirements for the Space 
Station Electric Power System (EPS). Vitro engineers have identified potential 
instabilities caused by the inadequate specification of load impedance characteristics. 
While the EPS design for Space Station Freedom is being changed for the International 
Space Station Alpha, the same problems exist with the impedance specifications. 

Support for this work will continue during the transition of the support effort to NASA. 

Expendable Launch Vehicles/Upper Stages, and Spacelab 

Vitro engineers have participated in the full range of design, flight readiness, and range 
safety reviews for the many ELV/US and Spacelab missions flown during this contract. 
Significant areas of achievement include assisting Code QP in the coordination of the 
commercial ELV launch SRM&QA policy with DOT, USAF and NASA, and the 
commercial ELV community. Vitro engineers were instrumental in gaining program and 
Code Q support for the Laser Initiated Ordnance program. Their technical expertise 
was used to support the special teams formed to investigate the TOS ACTS Super*Zip 
Anomaly on STS-51 and the Tethered Satellite Anomaly on STS-46. Again, their 
technical expertise and the exceptional technical support to Code QP has been noted in 
letters of commendation. 

Vitro engineers have formed a process improvement team to address the long-term and 
flight-to-flight issues associated with each type of ELV program. The result of their 
efforts is the Safety and Mission Assurance Review Telecon (SMART) Operating Plan. 
This plan recommends a process to Code QP for reviewing significant SRM&QA issues 
and concerns for each ELV mission, similar to the Space Shuttle Prelaunch Assessment 
Review. The review will include the NASA and industry ELV SRM&QA community to 
discuss issues related to both the specific mission, the launch vehicle, and the launch 
facility. The intent is to aid the SRM&QA community in coordinating process 
improvements that will contribute to mission success. 

Payloads and Missions 

At the time the payloads and mission programs transferred to the Quality Management 
(Payloads) Division (Code QT), the Vitro engineering support had been providing 
detailed, rapid-response, technical assessments of SRM&QA issues and concerns. Also, 
the engineers reviewed and recommended changes to NASA documents, such as NASA 
contract SRM&QA requirements, to enhance the effectiveness of SRM&QA 
requirements imposed on their Payloads programs. Their recommendations spanned the 
full range of the Payloads program design and mission life cycle, from the Phase A/B 
conceptual design processes to the flight performance. 

Since the transfer of the Payloads programs to Code QT, the Vitro engineers assigned to 
support Code QT have assisted in the development of a Cost of Quality seminar. 
Engineers on the Code QP task supported that effort, with the intent of expanding the 
emphasis beyond Payload programs to include the Space Shuttle, Space Shuttle/Mir, 
International Space Station Alpha, and Spacelab programs. 
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Aeronautics and Space Exploration Programs 

Significant efforts for this program area include assisting Code QP in the establishment 
of SRM&QA requirements within the Code C Centers for the Commercial Development 
of Space (CCDS) Programs, and establishing safety requirements for commercial 
customers flying experiments on the Space Shuttle. Vitro engineers have assisted Code 
QP in establishing SRM&QA requirements for the Shuttle/MIR program with their 
Russian Space Agency counterparts. Significant technical issues have been identified 
with the docking dynamics between the Space Shuttle and the Mir Space Station, with 
the Space Shuttle plume impingement, and with the Mir attitude control system. These 
issues are being worked as part of the Shuttle/Mir programs effort, and will continue to 
be worked as part of the new contract. Vitro’s unique technical support for these issues 
has been recognized by Code QP. 

Special Program Support 

As part of this contract, and part of the Payloads program effort for this task, Vitro 
provided special optical engineering quality assurance support to the NASA Associate 
Administrator for Safety, Reliability, Maintainability, and Quality Assurance, Code Q, 
during his participation on the Hubble Space Telescope Board of Investigation (also 
know as the Allen Committee). The Board was formed to determine how the error was 
made in the Space Telescope’s Primary Mirror during its manufacturing process, and 
remained undetected until the telescope was placed into orbit. Vitro supplied 
engineering and management support for this effort, and contributed to the "SRM&QA 
Observations and Lessons Learned" report published in October 1990. The optical 
support continued during the redesign effort and First Servicing Mission flown on STS-61 
in December 1993 to correct the optics. Due to the successful quality assurance efforts 
of the Vitro optical subcontractor, the Advanced X-Ray Astrophysics Facility (AXAF) 
program has requested Code Q to provide similar independent optical quality assurance 
for their program. 


III. CONCLUSIONS AND RECOMMENDATIONS 

While the requirements for technical and engineering support for the programs will 
remain, the direction within NASA has been to work more closely with each program to 
solve process problems. Vitro has recommended and will continue to recommend 
recurrence control for process problems as they are identified by our technical 
evaluations and assessments. NASA should incorporate the Space Shuttle Enhancement 
Review program approved during this task period, as time permits. Due to the fact that 
the Space Shuttle Program is a mature program, costs savings can only result by 
reviewing the SRM&QA processes to determine which processes provide the best value, 
and changing those that provide the least. Similarly, the SMART program for the ELV 
programs should be implemented to provide a forum for discussing mutual SRM&QA 
issues within the ELV SRM&QA community. Vitro engineers, in conjunction with the 
Division Director, have been drafting an Operating Plan for the Space Shuttle Program 
to provide a forum within Code Q for discussing and coordinating all of the RTOP 
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programs and field installation programs that impact that program. With the diverse 
efforts currently being performed within Code Q, and a need to reduce program costs 
while providing a complete program of support for the Space Shuttle Program, this 
forum will provide a process for advocating SRM&QA changes to the Program Office. 
Quality products such as the Cost of Quality seminar should be expanded to programs 
other than Payloads. The real value of the Cost of Quality concept is that it 
complements the Continuous Process Improvement techniques, while providing a process 
to focus improvements based on useful metrics. 
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10000 SERIES TASK ORDER 

SPACE STATION FREEDOM PROGRAM 

SAFETY AND PRODUCT ASSURANCE SUPPORT 


I. BACKGROUND FOR THE OVERALL TASK 

The NASA Space Station Freedom Program Office Level II Safety and Product 
Assurance Office (DSQ) responsibility was for the planning, direction, implementation, 
and evaluation of SSFP systems assurance including systems safety, reliability, 
maintainability, and quality assurance including software product assurance. DSQ 
provided for overall independent technical review of SSFP/projects to ensure 
development efforts and mission operations were being conducted on a sound 
engineering basis with proper controls and attention to development risk. In addition, 
DSQ performed review and oversight activities to help ensure that the design and 
operational procedures prepared for the SSF payloads and experiments were 
accomplished to preclude the occurrence of hazards that could cause loss of life or injury 
of the crew; cause loss of, or significant degradation to the SSF; and in normal 
operations, cause damage to other elements. DSO also was responsible to ensure that 
the Director and Deputy Director, SSFPO, and other principal officials were aware of 
SRM&QA matters pertaining to the technical execution and physical readiness of the 
SSFP and related projects. 

Vitro provided DSQ with the necessary technical and managerial skill and expertise in 
providing multi-discipline support in the principal functional areas of safety, reliability 
maintainability, and quality assurance (SRM&QA). 


II. SUMMARY OF TASK ACTIVITY 

This task series required Vitro to provide management, safety, reliability, quality 
assurance, and software produce assurance support. 

Management Support 

Vitro provided technical engineering support to perform assigned engineering tasks as 
directed by the Safety and Product Assurance Office Managers. Vitro helped to ensure 
that the Level II Space Station Freedom Program engineering activities complied with 
program requirements by providing superior independent technical assessment and 
oversight of the Level II /III S&PA guidance, direction, and priorities established by DSO 
managers. 

Vitro management was totally committed to the development and implementation of (1) 
a safety program that supported the challenges of safely establishing, operating and 
maintaining space station; (2) a reliability/maintainability program that identified, 
corrected, or controlled critical design weaknesses, repair requirements, and resources; 

(3) software defined and developed to a level of quality consistent with program 
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requirements and goals; and (4) a quality assurance program that verified compliance 
with the Space Station Freedom program requirements. 

Safety Support 

Vitro safety engineers supported evolution of SSF program safety documentation (SSP 
30000, Section 9, Program Definition and Requirements Document; SSP 30309, Safety 
Analysis and Risk Assessment Requirements; SSP 30599, SSFP Safety Review Process; 
SSP 30688, SSFP Integrated Safety Program Plan; SSP 30652, SSFP Payload Safety 
Requirements; SSP 30685, SSFP Hazard Data base System Requirements Definition 
Document) through participation in the development of major program safety 
requirements and review of draft documents to ensure comprehensive inclusion of 
programmatic safety requirements. Vitro safety engineers interacted with program 
participants to review and integrate draft modifications in response to programwide 
reviews, and prepared and presented CRs to the Integration Management Review 
(IMR), pre-Space Station Control Board (SSCB), and SSCB. Vitro also monitored 
progress of the documents through the SSFP review and approval process. 

The Vitro safety engineers performed independent Change Evaluations (CEs) of more 
than 300 CRs to modify existing Space Station documents or baseline new documents. 

In many instances, the safety engineers worked with the program engineers who were the 
proponents of CRs to revise them in an effort to remove any adverse safety impacts prior 
to presentation to the SSCB. The safety engineers performed independent assessments 
of Level II Systems Engineering and Integration safety products for the Space Station 
Safety Office. Chief among these were assessments of the Integrated Safety Preliminary 
Design Review (ISPDR) Integrated Safety Assessment Report, the Man-Tended 
Capability (MTC) Phase Review Integrated Safety Assessment Report, the Integrated 
Risk Assessment Reports for Stages 1 through 4, and the Program Incremental Design 
Review (PIDR-93). 

Vitro safety engineers actively participated in Space Station Freedom (SSF) level, Work 
Package (WP) level, prime contractor, Kennedy Space Center (KSC), and International 
Partner (IP) design reviews of flight hardware/software, ground support equipment, data 
systems, engineering support centers, and processing facilities. The safety engineers 
attended design presentations, reviewed design documentation, performed design 
assessments, developed Review Item Discrepancy (RID) reports, and participated on 
design review teams to disposition RIDs. At major milestone reviews (e.g., ISPDR, 

MTC Phase Review, and the Program Incremental Design Review), Vitro was 
responsible for performing independent global safety risk assessments. Using insights 
gained during participation in major milestone reviews, the safety engineers assisted the 
Safety Manager in developing safety risk assessments and presentations. Review 
participation included three major milestone reviews: the ISPDR and the MTC Phase 
Review and the Program Incremental Design Review. Other reviews supported included 
the WP-2 Distributed Systems Preliminary Design Review (PDR), the Canadian Space 
Agency (CSA) Interim Design Review (IDR), the WP-2 Delta PDR, the CSA Delta 
PDR, the KSC Ground Handling Equipment PDR, the KSC Core Electronics Contract 
PDR, the CSA Engineering Support Center PDR, the CSA Mobile Servicing System 
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Software Delta IDR, the WP-1 Delta PDR, the Central Software Facility/Central 
Avionics Facility System Requirements Review, the Communications and Tracking 
System Ground Support Equipment PDR, the Flight System Software Requirements 
Review, the National Space Development Agency (NASDA) Japanese Experimental 
Module PDR, the European Space Agency (ESA) Attached Pressurized Module System 
Requirements Review, and the CSA Space Station Remote Manipulator System CDR. 

The Vitro safety engineers assisted in the development and implementation of the 
Freedom Safety Review Panel (FSRP), which is still used to conduct independent safety 
reviews of the Space Station Manned Base (SSMB) configuration/operations for each 
stage of the SSMB construction and development. The safety engineers provided 
technical review of the Safety Compliance Data Packages (SCDP) from each WP or IP. 
Vitro safety engineers supported the FSRP chairman at each safety review by 
identifying/recording issues, concerns, and required actions; researching requirements; 
preparing summary reports, and assuring the accuracy of, and concurrence with, the 
minutes of the joint FSRP/Payload Safety Review Panel (PSRP) safety reviews. 

Vitro safety engineers supported development of safety associated SSF programwide data 
bases through participation in various Technical Management Information System 
(TMIS) Joint Application Development conferences and the conduct of programwide 
data base needs assessments. Safety engineers interfaced with program participants to 
establish data collection, system business processes, process work flow, business priorities, 
data needs, data flow and, used the information to develop chapters for the data base 
requirements definition document. The safety engineers conducted user acceptance 
testing of data base functionality and user interfaces and reviewed user documentation 
for accuracy and comprehensiveness. 

Vitro safety engineers actively supported the Space Station Safety Office in the conduct 
of four major safety-related trade studies. The studies addressed Space Station 
emergency egress scenarios, module arrangement, and translation paths; manual power 
switches to control electrical power to module equipment and payload racks; proper 
settings for ground fault interrupters; and fire detection and suppression system 
components definition and locations. The studies required extensive coordination with 
Space Station Work Package 1, 2, and 4 engineers, IPs, contractors, consultants, industry 
sources, and NASA activities. 


Reliability and Maintainability Support 

The Vitro Reliability and Maintainability (R&M) team was deeply involved in the 
attainment of mission success and the goal to assure that the Space Station will be 
reliable and maintainable over its life span. Vitro R&M engineers participated in every 
facet of the Space Station program — from basic requirements definition and the 
generation of plans to very detailed engineering analysis through the use of mathematical 
simulation models. Vitro provided R&M engineering capabilities in direct support 
providing rapid response to many problems and action requests. Vitro R&M engineers 
demonstrated an innate sense of responsibility and urgency, which resulted in an 
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appreciation of teamwork, continuous process improvement, and interactive participation 
required for a meaningful SSF product assurance program. Specific examples of direct 
SSF program participation included: 

• Submission of many Review Item Discrepancies for R&M requirements which 
surfaced at the Integrated System Preliminary Design Review, the Man Tended 
Capability Phase Preliminary Design Review, and the Program Incremental Design 
Review-93. 

• Submission of CRs for the Program Definition and Requirements Document, 
Section 9 Data Requirements and baselining of the Level II R&M Program Plan, 
evaluation of numerous CRs dealing with design requirements; and complete 
traceability of R&M requirements from SSP 30000, Sections 3 and 9 down to Level 
III Contract End Item specifications^ 

• Documentation and planning tasks, including the total development of a 
comprehensive Reliability & Maintainability Program Plan (TSS 30579) for Level 
II; development of S&PA support plans for the ISPDR, MTC Phase PDR, and 
MTC CDR. 

• Closure of many action items dealing with subjects as diverse as the CSA ground 
segment and design review traceability; development of an R&M data base for 
audits, findings, observations, and corrective actions; development of the Level II 
S&PA Verification Assessment Plan to be included as a volume of the Program 
Master Verification Plan, SSP 30666, and development of a NASA Handbook for 
the preparation of FMEA/CILs, which if implemented will affect every NASA 
program. 

Work performed by the R&M team in an oversight role include team membership and 
leadership during Level II, Level HI, and IP technical milestone and design reviews; 
audits and R&M data reviews at the WPs and IPs; IP meet-or-exceed reviews and data 
exchange agreements; verification assurance; analysis of the SSF two-failure-tolerant 
attitude control function design; training in the Failure Modes and Effects 
Analysis/Critical Items List (FMEA/CIL) Data Base for review and analysis of failures; 
and S&PA representation in the working group which analyzed the various DC-to-DC 
Converter Unit location options. 

Management activities included active participation in the Redundancy Management 
Panel; program, DSQ and R&M stand-up meetings; Electrical, Electronic 
Electromagnetic (EEE) Parts Advisory Board, Program Verification Panel; Mission 
Integration Review group; PDR and CDR planning groups; model development and 
analysis working group; TMIS data base and application meetings, including the Failure 
Environment Analysis Tool/Digraph Development Group; and the program SRM&QA 
Managers’ teleconferences. 

Vitro’s technical analytic endeavors include the performance of availability modeling 
through the construction of reliability block diagrams and the calculation of availability 
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figures for SSF survival functions using a Monte Carlo simulation model, the assessment 
of changes in availability due to redesign changes in MTBF and MTTR, and the 
identification of weak links in the design. The Event Time Availability Reliability 
Analysis (ETARA) model was used to calculate the reliability and availability of the 
early Space Station Freedom Stages (1-6). The purpose was to make design tradeoffs for 
a number of configurations of the survival functions and stage durations. It was found, 
for example, that maintenance flights would be needed if Stage durations exceeded 3 
months, and that reliability was significantly enhanced by using actively cooled dc-dc 
converters. 

Additionally, Vitro R&M engineers participated in a select team evaluation of SSF 
R&M data bases, and made the recommendation that all program offices utilize a 
consistent method to assess Orbital Replacement Unit failure rates, life limits, and repair 
times. Vitro R&M engineers performed an independent assessment of the SSF Master 
"Verification Requirements and the Verification Responsibility Matrix. They also actively 
participated in the KSC Cargo Element Lifting Assembly CDR, the delta Software 
Requirements Review for the SSF Central Software Facility and Central Avionics 
Facility, the Ground Systems Program Review, the KSC SSF Processing Facility 
Ammonia Servicing System, the KSC Removable Overhead Access Platform, the ESA 
System Requirements Review, the NASDA Japanese Experiment Module PDR, and the 
CSA Engineering Support Center CDR. 

Vitro R&M engineers interfaced with the Space Station IPs on a frequent basis. We 
participated in the European Space Agency Preliminary Requirements Review at the 
ESA Technical and Engineering Center in Noordwijk, in the ESA/NASA meets-or- 
exceeds meetings, with National Space Development Agency of Japan in the Redundancy 
Management Panel, and with CSA at numerous requirements and design reviews of the 
Space Station Remote Manipulator and the Special Purpose Dextrous Manipulator. The 
R&M section has many contacts within the IPs organizations which allows it to have 
quick and effective communications with the IPs. 

The Vitro R&M support team had extensive interface in the SSF program, including 
organization, levels, management structure, and a working relationship with key 
personnel at NASA Headquarters Code Q, all the Work Package centers and prime 
contractors, the Space Station Engineering and Integration Contractor (SSEIC), KSC, 
and the IPs. SSF experience included hands-on development and analyses dealing with 
requirements, documents, controls, and configuration management procedures. The team 
has thorough knowledge of the phased process of the SSF development, design review, 
production, acceptance, verification, launch-to-orbit, and operation of the SSF over its 
life span. 

The entire reliability, maintainability, and verification assurance programs, including 
those tasks of the Systems Engineering and Integration group, are well known, as is the 
establishment of the requirements of SSP 30234, the FMEA/CIL requirements for Space 
Station program, and the FMEA/CIL assessment activity in major design reviews. This 
unique knowledge permitted Vitro to establish a history of proven R&M 
accomplishments across the full spectrum of R&M engineering. 
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Quality Assurance Support 

Vitro Quality Assurance (QA) engineers supported the evolution of the SSF design and 
program documentation through the continuous evaluation, assessment and independent 
oversight of requirements traceability to the manufacturer level; Engineering Change 
Packages (ECPs) and CRs; design presentations and design data packages; technical 
documentation; data system integrity and capabilities; and WP Centers/IPs/Prime 
Contractors/KSC operations and procedures. 

Vitro QA engineers supported the goal of advancement of world class QA principles and 
techniques by developing and maintaining QA program requirements through the 
initiation of program CRs and formulation of CEs). The QA Engineers supported the 
SSFPO S&PA Office in the maintenance of the following program baselined documents 
for which the office is the book manager: 

• SSP 30000, Section 9, "Program Definition Requirements Document." 

• SSP 30223 "Problem Reporting and Corrective Action System Requirements for the 
Space Station Program." 

• SSP 30521 "Safety and Product Assurance Audit Survey Plan." 

• SSP 30523 Safety and Product Assurance Information Planning Group (IPG) 
Overview Document." 

• SSP 30524 "Problem Reporting and Corrective Action (PRACA) System 
Requirements Definition Document (RDD). 

Vitro QA engineers actively and intensely participated in SSFP, WP Center, Prime 
Contractor, KSC, and IP design reviews of flight hardware/software, ground support 
equipment, data systems, engineering support centers, and processing facilities. The QA 
staff attended design presentations, performed assessment of designs, reviewed design 
documentation, developed RID reports, and participated on design review teams to 
disposition RIDS. Through these activities, the QA staff ensured that program QA 
requirements are incorporated throughout program/center/company specifications and 
standards. To enhance design review productivity, the QA staff utilized the Automated 
Requirements Management System (ARMS) to verify requirements traceability and the 
Automated RID Tracking System (ARTS) to track RID dispositions. 

Vitro QA engineers ensured that program requirements were incorporated in and 
properly flowed down to all applicable lower level documents, standards, and 
specifications. Vitro engineers supported the Configuration Management Office, as a 
representative of the S&PA Office, on all Requirements Traceability Audits. 
Requirements traceability was also accomplished during evaluations of CRs and by 
participation in design reviews and various working groups. Additionally, the QA staff 
ensured that all new requirements were verifiable and that the requirements identified in 
ARMS were incorporated in the Master Verification Data Base (MVDB). 
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Vitro QA engineers performed special assessments and analyses upon request of the 
S&PA Office. The QA staff assessed the adequacy of the records retention program of 
all SSFP Participants (SSFPPs); identified issues with the definition of high strength 
fasteners throughout NASA and the aerospace industry; performed comparisons of 
terminology between the NASA SSFP and the ESA Columbus Program; conducted 
reviews of NASA workmanship standards; assessed the quality program of WP Centers in 
preparation for S&PA audits; identified issues with the use of conflicting units 
conversions by NASA and IP; assessed the adequacy and implementation of 
Fastener/Quick Disconnects/Electrical Connectors requirements on the program; 
evaluated the compatibility of NHB 5300.4 (ID-2) and SSP 30000 Section 9, Chapter 4.0 
requirements; and reviewed numerous engineering analyses. Vitro QA engineers actively 
participated in the QA Technical Interchange Meetings (TIMs) and presented results 
and overviews of its subject activities. 

Vitro QA engineers supported development and maintenance of QA associated SSF 
programwide data bases through participation in Joint Application Development (JAD) 
conferences, TMIS Application Technical Reviews, TMIS Branch level reviews, TMIS 
Operations Readiness Reviews, TMIS Control Boards, and various other working groups. 
Vitro QA engineers worked with program participants to establish data collection system 
business processes, process work flow, business priorities, data element needs, and data 
requirements. In conjunction with the NASA Office of Primary Responsibility (OPR), 
IBM, Boeing, and TMIS, Vitro QA engineers developed user scenarios, data models, 
entity relationship diagrams, process decomposition diagrams, and data tables. Vitro QA 
engineers conducted user acceptance testing of data base functionality and user 
interfaces and reviewed user documentation for accuracy and comprehensiveness. Vitro 
QA engineers evaluated the Requirements Definition Documents for most TMIS data 
systems for S&PA related data processing impacts and acted as the representative for the 
S&PA Office on the development of the Test, Operations, Maintenance Requirements 
and Specifications (TOMRS) system. 

Vitro QA engineers supported the NASA OPR on the development, evolution, and 
maintenance of the Problem Reporting and Corrective Action (PRACA) System and 
Data System (PDS). The Vitro QA engineers identified and defined user requirements, 
performed system assessments, conducted user acceptance testing, assisted in 
development of CRs, assisted in disposition of CEs, evaluated design approaches, and 
coordinated activities of the PRACA development team. Vitro QA engineers 
coordinated and participated in PRACA JAD conferences, PRACA Process 
Improvement Team (PPIT) meetings, PRACA Design Review Team (DRT) meetings, 
and various telecons. Vitro QA engineers assisted in the maintenance and evolution of 
SSP 30223, "Problem Reporting and Corrective Action System Requirements for the 
Space Station Program," and SSP 30524, "Problem Reporting and Corrective Action 
(PRACA) System Requirements Definition Document (RDD)." Vitro QA engineers 
supported the NASA OPR at the TMIS Control Board (TCB) and SSCB and played an 
important role in PRACA meet-or-exceed evaluations of the NASDA, ESA, and CSA. 
Vitro QA engineers performed extensive data element comparisons and business process 
assessments for the meet or exceed evaluations for both the NASDA and ESA. Vitro 
QA engineers also played a significant role in assessing the feasibility of adapting and 
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converting the Johnson Space Center (JSC) Space Shuttle (SS) PRACA process to utilize 
the SSFP PDS. Vitro QA engineers assisted in the development of Nonconformance 
Reporting (NCR) capabilities, In-Flight Anomaly (IFA) reporting, and Corrective Action 
Assistance Request (CAAR) processing capabilities in the SSFP PDS. Vitro QA 
engineers supported the development of future capabilities, such as problem trending 
and analysis. 

The Vitro QA engineers supported verification activities through the review of the 
program verification requirements in SSP 30000, Section 12, "Space Station Program 
Master Verification Requirements"; SSP 30467, Volume II, "Space Station Program 
Master Verification Requirements: Master Verification Implementation Requirements"; 
and SSP 30468, Volume II, "Space Station Combined Elements and Integrated Systems 
Process Requirements: Combined Elements and Integrated Systems Verification 
Implementation Requirements." Through various means, Vitro QA engineers assured 
that verification responsibilities are considered and planned by all program participants. 
Vitro QA engineers supported various verification working groups and meetings such as 
the Program Master Verification Requirement (PMVR) review and the Stage 
Verification Working Group. 

Software Product Assurance 

The Vitro Software Product Assurance (SPA) engineers actively supported evolution of 
SSF program software documentation based on requirements of SSP 30000, Program 
Definition and Requirements Document, Section 9, Product Assurance Requirements, 
Paragraph 5, Software Product Assurance, in support of the SSF Level II SPA office. 

This was accomplished through participation (via independent assessment) in the 
development and implementation of software requirements on the program, and the 
review of draft documents to ensure traceability of programmatic S&PA requirements. 
Vitro SPA engineers assured that software product assurance concerns were addressed in 
software documentation. Vitro SPA engineers coordinated with other SSF directorate 
activities, including IPs, to review and integrate changes to requirements, design, and 
verification/ test documentation. Vitro SPA engineers performed research using various 
documents from NASA, American Institute Aeronautics and Astronautics, Federal 
Aviation Administration, Institute of Electrical and Electronics Engineers, private 
industry etc., in order to utilize the latest "state-of-the-art" techniques and information. 
The Level II SPA activity was recognized as the software management focal point for 
issues related to software safety, requirements verifiability, and other issues associated 
with software. Vitro SPA engineers established sound working relationships through 
interface and coordination with Level I, Level II, WP, IP, and contractor and 
subcontractor personnel, to derive ‘real issues’ and bring them to management’s 
attention. 

Specific examples of direct SSF program S&PA participation include SSF program 
reviews and performance of review-related tasks, including: 

• Evaluation of program requirements and IP "meet-or-exceed" requirements. 
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• Proactive engineering support in Requirements Traceability Audits conducted for 
the purpose of evaluating traceability or flowdown of requirements to the Work 
Package Centers and CSA and their major contractors and the submittal of Audit 
Evaluation Reports. 

• Engineering assessment of Level A Integrated Flight Software Architecture 
Requirements to assure that the Fault Detection Identification and Reconfiguration 
requirements meet the criteria for catastrophic and critical hazards for time-critical 
events. 

• Participation in industry and government research related to Software Reliability; 
provided suggestions to the program on a consistent approach for software 
reliability on the program. 

• Active involvement in the Software Fault Analysis Working Group and the 
finalization of the Software Fault Analysis Plan. 

• Evaluation and review of major contractor SPA plans for compliance with 
requirements in Section 9, DR SSQ-SPA-001. 

• Active engagement in TIMs with NASA Headquarters and WP Centers. 

Vitro SPA engineers were actively engaged in the NASDA Software Design Review and 
developed RIDs in support of the program effort. Especially of note was the RID that 
precipitated the NASDA SPA TIM, involving the identification of deficiencies in the 
NASDA SPA plan that did not meet the intent of DR SSQ-SPA-001. 

Vitro SPA engineers were actively involved in the development, evaluation, or support of 
ongoing engineering efforts in CFR; Level II SPA plan; safety definitions working group; 
PRACA software non-conformance issues; software metrics; Ada coding standards; and 
scope of Independent Verification and Validation (IV&V) activities. 

Vitro SPA engineers supported all NASA software assurance activities, including 
developing documents, defining assurance activities, and supporting the NASA Software 
Steering Committee. Documents that Vitro S&PA had a significant role in developing 
included NMI 2410.10, "NASA Software Management, Assurance and Engineering 
Policy"; NASA-STD-2201-93, "Software Assurance Standard; SMPA-GB-A201, Software 
Assurance Guidebook"; and SMAP-GB-A301, "Software Quality Assurance Audits 
Guidebook." Vitro also participated in the development of the Implementation Plan for 
the IV&V Facility in Fairmont, WV. 


III. CONCLUSIONS AND RECOMMENDATIONS 

It is recommended that NASA Headquarters develop, in conjunction with all NASA 
Centers, a set of Agencywide Safety, Reliability, Maintainability, and Quality Assurance 
(SRM&QA) requirements documents or handbooks that would apply to every NASA 
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program, project, and organization. The requirements document should standardize 
every aspect of the total SRM&QA program but yet allow for tailoring or deviating in 
certain areas for special conditions. The Agencywide requirements concept would 
eliminate the need to develop requirements for each NASA program and allow 
personnel to easily work across programs. 
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